Hyperledger Fabric之BYFN多机部署
文章目录安装HLF配置docker网络创建启动文件运行可能出现的错误
作者：yijiull
显然单机和多机部署的区别在于网络配置，也就是怎么进行docker容器之间的通信，所以多机部署的关键其实在于docker的网络通信。
安装HLF五台server，系统是Ubuntu16.04，已经配置好了ssh证书登录，假设IP分别是
10.22.1.12 # orderer.example.com, cli
10.22.1.13 # peer0.org1.example.com
10.22.1.14 # peer1.org1.example.com
10.22.1.15 # peer0.org2.example.com
10.22.1.16 # peer1.org2.example.com
如果没有特殊说明，以下命令均在10.22.1.12机器上/home/myuser/目录下进行。
创建一个host.txt文件，方便动态修改部署到其它机器：
vim hosts.txt
10.22.1.12 # orderer.example.com, cli
10.22.1.13 # peer0.org1.example.com
10.22.1.14 # peer1.org1.example.com
10.22.1.15 # peer0.org2.example.com
10.22.1.16 # peer1.org2.example.com
安装curl、docker、docker-composer、HLF镜像（1.3的版本，如需其它版本自行替换脚本中的版本号）的脚本install.sh，需要用户密码作为第一个参数：
#!/bin/bash
if [ $# -eq 0 ]; then
echo "please input password..."
exit 1
fi
cat hosts.txt | while read host
do
echo ' '
echo ' '
echo ' '
echo '################################'
echo '##########' $host '##########'
echo '################################'
echo ' '
echo ' '
echo ' '
ssh myuser@$host << end
echo $1 | sudo apt install curl -y
sudo -S apt-get remove docker docker-engine docker.io containerd runc -y
docker -v
sudo apt-get update -y
sudo apt-get install \
apt-transport-https \
ca-certificates \
curl \
gnupg2 \
software-properties-common -y
curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -
sudo add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/debian \
$(lsb_release -cs) \
stable"
sudo apt-get update -y
sudo apt-get install docker-ce docker-ce-cli containerd.io -y
docker run hello-world
docker -v
if [ $? -ne 0 ]; then
echo "##################################################"
echo "some wrong when install docker!"
echo "##################################################"
exit 1
fi
echo $1 | sudo -S curl -L "https://github.com/docker/compose/releases/download/1.25.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
sudo rm /usr/bin/docker-compose
sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
docker-compose -v
if [ $? -ne 0 ]; then
echo "##################################################"
echo "some wrong when install docker-compose!"
echo "##################################################"
exit 1
fi
curl -sSL http://bit.ly/2ysbOFE | bash -s 1.3.0
if [ $? -ne 0 ]; then
echo "##################################################"
echo "some wrong when pull the images"
echo "##################################################"
exit 1
fi
end
done
echo done!
到这里不出意外的话就算是配置好了，可以到/home/myuser/fabric-samples/first-network/目录下测试是否安装成功：
启动网络：
./byfn.sh up
如果看到下面的输出就是安装成功了：
90
===================== Query successful on peer1.org2 on channel 'mychannel' =====================
========= All GOOD, BYFN execution completed ===========
_____ _ _ ____
| ____| | \ | | | _ \
| _| | \| | | | | |
| |___ | |\ | | |_| |
|_____| |_| \_| |____/
关闭并清理网络（每次重启网络前都要记得清理，否则会报错）：
./byfn.sh down
配置docker网络
据说可以使用host模式进行部署，简单很多，但是我使用host模式时在实例化chaincode的时候总是报错，没找到解决方案，于是采用了overlay模式。
docker swarm init --advertise-addr=10.22.1.12 # 在10.22.1.12上初始化swarm
docker swarm join-token manager # 这里会输出一个命令docker swarm join --token ... ，复制下来，去其他机器上执行
ssh myuser@10.22.1.13
docker swarm join --token ...
ssh myuser@10.22.1.14
docker swarm join --token ...
ssh myuser@10.22.1.15
docker swarm join --token ...
ssh myuser@10.22.1.16
docker swarm join --token ...
# 回到10.22.1.12
docker network create --attachable --driver overlay HLF # 创建一个overlay网络
这时查看docker的网络docker network ls，大概会是这样：
NETWORK ID NAME DRIVER SCOPE
ntubodu3k0fp HLF overlay swarm
31158df52877 bridge bridge local
54ad61772123 docker_gwbridge bridge local
d8b38ea6fbed host host local
rlzokyfpla8r ingress overlay swarm
e97a070185d2 none null local
其他机器上也是一样，它们会自动创建。
创建启动文件在/home/myuser/fabric-samples/first-network目录下有一个docker-compose-cli.yaml文件，主要根据这个文件进行修改。
将这个文件分成5个，一个包含orderer和cli，另外四个分别是四个peer。
.
├── base
│ ├── docker-compose-base.yaml
│ └── peer-base.yaml
├── docker-compose-cli.yaml
├── orderer_cli.yaml
├── peer01.yaml
├── peer02.yaml
├── peer11.yaml
└── peer12.yaml
主要修改内容是将networks修改为我们之前创建的网络HLF，不过需要注意指定external为true，否则docker会自动重新创建一个网络。下面是全部的文件：
orderer_cli.yaml:
# Copyright IBM Corp. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0
#
version: '2'
volumes:
orderer.example.com:
networks:
HLF:
external: true ## important!!!!!
services:
orderer.example.com:
extends:
file: base/docker-compose-base.yaml
service: orderer.example.com
container_name: orderer.example.com
#network_mode: overlay
networks:
- HLF
# extra_hosts:
# - "orderer.example.com:10.22.1.12"
# - "peer0.org1.example.com:10.22.1.13"
# - "peer1.org1.example.com:10.22.1.14"
# - "peer0.org2.example.com:10.22.1.15"
# - "peer1.org2.example.com:10.22.1.16"
cli:
container_name: cli
image: hyperledger/fabric-tools:$IMAGE_TAG
tty: true
stdin_open: true
environment:
- GOPATH=/opt/gopath
- CORE_VM_ENDPOINT=unix:///host/var/run/docker.sock
#- CORE_LOGGING_LEVEL=DEBUG
- CORE_LOGGING_LEVEL=INFO
- CORE_PEER_ID=cli
- CORE_PEER_ADDRESS=peer0.org1.example.com:7051
- CORE_PEER_LOCALMSPID=Org1MSP
- CORE_PEER_TLS_ENABLED=true
- CORE_PEER_TLS_CERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/server.crt
- CORE_PEER_TLS_KEY_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/server.key
- CORE_PEER_TLS_ROOTCERT_FILE=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/tls/ca.crt
- CORE_PEER_MSPCONFIGPATH=/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/peerOrganizations/org1.example.com/users/Admin@org1.example.com/msp
working_dir: /opt/gopath/src/github.com/hyperledger/fabric/peer
command: /bin/bash
volumes:
- /var/run/:/host/var/run/
- ./../chaincode/:/opt/gopath/src/github.com/chaincode
- ./crypto-config:/opt/gopath/src/github.com/hyperledger/fabric/peer/crypto/
- ./scripts:/opt/gopath/src/github.com/hyperledger/fabric/peer/scripts/
- ./channel-artifacts:/opt/gopath/src/github.com/hyperledger/fabric/peer/channel-artifacts
networks:
- HLF
# extra_hosts:
# - "orderer.example.com:10.22.1.12"
# - "peer0.org1.example.com:10.22.1.13"
# - "peer1.org1.example.com:10.22.1.14"
# - "peer0.org2.example.com:10.22.1.15"
# - "peer1.org2.example.com:10.22.1.16"
peer01.yaml:
# Copyright IBM Corp. All Rights Reserved.
# SPDX-License-Identifier: Apache-2.0
#
version: '2'
volumes:
peer0.org1.example.com:
networks:
HLF:
external: true
services:
peer0.org1.example.com:
container_name: peer0.org1.example.com
extends:
file: base/docker-compose-base.yaml
service: peer0.org1.example.com
#network_mode: overlay
#networks:
# - byfn
networks:
- HLF
peer02.yaml:
# Copyright IBM Corp. All Rights Reserved.
# SPDX-License-Identifier: Apache-2.0
#
version: '2'
volumes:
peer0.org1.example.com:
networks:
HLF:
external: true
services:
peer0.org1.example.com:
container_name: peer0.org1.example.com
extends:
file: base/docker-compose-base.yaml
service: peer0.org1.example.com
#network_mode: overlay
#networks:
# - byfn
networks:
- HLF
peer11.yaml:
# Copyright IBM Corp. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0
#
version: '2'
volumes:
peer0.org2.example.com:
networks:
HLF:
external: true
services:
peer0.org2.example.com:
container_name: peer0.org2.example.com
extends:
file: base/docker-compose-base.yaml
service: peer0.org2.example.com
#network_mode: overlay
#networks:
# - byfn
networks:
- HLF
# extra_hosts:
# - "orderer.example.com:10.22.1.12"
# - "peer0.org1.example.com:10.22.1.13"
# - "peer1.org1.example.com:10.22.1.14"
# - "peer0.org2.example.com:10.22.1.15"
# - "peer1.org2.example.com:10.22.1.16"
peer12.yaml:
# Copyright IBM Corp. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0
version: '2'
volumes:
peer1.org2.example.com:
networks:
HLF:
external: true
services:
peer1.org2.example.com:
container_name: peer1.org2.example.com
extends:
file: base/docker-compose-base.yaml
service: peer1.org2.example.com
#network_mode: overlay
# networks:
# - byfn
networks:
- HLF
# extra_hosts:
# - "orderer.example.com:10.22.1.12"
# - "peer0.org1.example.com:10.22.1.13"
# - "peer1.org1.example.com:10.22.1.14"
# - "peer0.org2.example.com:10.22.1.15"
# - "peer1.org2.example.com:10.22.1.16"
除此之外还需要修改/home/myuser/fabric-samples/first-network目录下的两个文件：
base
├── docker-compose-base.yaml
└── peer-base.yaml
peer-base.yaml中的- CORE_VM_DOCKER_HOSTCONFIG_NETWORKMODE=${COMPOSE_PROJECT_NAME}_byfn修改为- CORE_VM_DOCKER_HOSTCONFIG_NETWORKMODE=HLF。
docker-compose-base.yaml中的端口映射部分：
- 7051:7051
- 7053:7053
- 8051:7051
- 8053:7053
- 9051:7051
- 9053:7053
- 10051:7051
- 10053:7053
修改为
- 7051:7051
- 7053:7053
- 7051:7051
- 7053:7053
- 7051:7051
- 7053:7053
- 7051:7051
- 7053:7053
到此修改完成。
这时可以参考官网的命令去生成证书文件和通道配置文件，脚本如下：
#!/bin/zsh
cd /home/myuser/fabric-samples/first-network
# clear the old X.509
rm -rf crypto-config
rm -rf channel-artifacts/*
# generate X.509 using the default configuration
cryptogen generate --config=crypto-config.yaml
# tell the tool where to look for the configtx.yaml
export FABRIC_CFG_PATH=$PWD
export CHANNEL_NAME=mychannel
# create the orderer genesisi block
configtxgen -profile TwoOrgsOrdererGenesis -outputBlock ./channel-artifacts/genesis.block #-channelID $CHANNEL_NAME
# create a channel configuration transaction
configtxgen -profile TwoOrgsChannel -outputCreateChannelTx ./channel-artifacts/channel.tx -channelID $CHANNEL_NAME
# define the anchor peer for Org1
configtxgen -profile TwoOrgsChannel -outputAnchorPeersUpdate ./channel-artifacts/Org1MSPanchors.tx -channelID $CHANNEL_NAME -asOrg Org1MSP
# define the anchor peer for Org2
configtxgen -profile TwoOrgsChannel -outputAnchorPeersUpdate ./channel-artifacts/Org2MSPanchors.tx -channelID $CHANNEL_NAME -asOrg Org2MSP
将修改的7个yaml文件和证书crypto-config通过scp传送到对应的机器，对应的位置。
启动脚本：./run.sh
#!/bin/bash
peer01=10.22.1.13
peer11=10.22.1.14
peer02=10.22.1.15
peer12=10.22.1.16
cd ~/fabric-samples/first-network
docker-compose -f orderer.yaml up -d
ssh myuser@$peer01 "cd fabric-samples/first-network; docker-compose -f peer01.yml up -d"
ssh myuser@$peer11 "cd fabric-samples/first-network; docker-compose -f peer11.yml up -d"
ssh myuser@$peer02 "cd fabric-samples/first-network; docker-compose -f peer02.yml up -d"
ssh myuser@$peer12 "cd fabric-samples/first-network; docker-compose -f peer12.yml up -d"
进入cli容器：docker exec -it cli bash
在cli容器中执行脚本进行测试: bash scripts/script.sh
看到如下输出说明成功：
90
===================== Query successful on peer1.org2 on channel 'mychannel' =====================
========= All GOOD, BYFN execution completed ===========
_____ _ _ ____
| ____| | \ | | | _ \
| _| | \| | | | | |
| |___ | |\ | | |_| |
|_____| |_| \_| |____/
可能出现的错误
拉取镜像的时候，可能会报curl: (56) recv failure: connection reset by peer：将install.sh中的curl -sSL http://bit.ly/2ysbOFE | bash -s 1.3.0 改为 curl -sSL https://bit.ly/2ysbOFE | bash -s 1.3.0 即可
在cli容器中执行脚本进行测试的时候可能会有各种情况，这个时候就先检查网络是否连通，然后检查证书文件是否有问题，…。
作者：yijiull
