Hyperledger Fabric 部署在多个主机上

前言

在实验Hyperledger Fabric无排序组织以Raft协议启动多个Orderer服务、TLS组织运行维护Orderer服务中，我们已经完成了使用提供 TLS-CA 服务的 council 组织运行维护 Raft 协议的三个 orderer 节点。但目前我们都是在单个主机上启动 Fabric 网络，本文将尝试将 Hyperledger Fabric无排序组织以Raft协议启动多个Orderer服务、TLS组织运行维护Orderer服务中的网络结构部署在多个主机上。

工作准备

本文工作

将 Hyperledger Fabric无排序组织以Raft协议启动多个Orderer服务、TLS组织运行维护Orderer服务中网络部署至两台主机上—— DebianA 和 DebianB，其中 DebianA 维护 council 和 soft 组织及相关节点， DebianB 维护 web 和 hard 组织及相关节点，网络结构为（实验代码已上传至：https://github.com/wefantasy/FabricLearn 的 5_FabricNetworkByMultiHost 下）：

项	所属主机	运行端口	说明
`council.ifantasy.net`	DebianA	7050	council 组织的 CA 服务，为联盟链网络提供 TLS-CA 服务
`orderer1.council.ifantasy.net`	DebianA	7051	orderer1 的排序服务
`orderer1.council.ifantasy.net`	DebianA	7052	orderer1 的 admin 服务
`orderer2.council.ifantasy.net`	DebianA	7054	orderer2 的排序服务
`orderer2.council.ifantasy.net`	DebianA	7055	orderer2 的 admin 服务
`orderer3.council.ifantasy.net`	DebianB	7057	orderer3 的排序服务
`orderer3.council.ifantasy.net`	DebianB	7058	orderer3 的 admin 服务
`soft.ifantasy.net`	DebianA	7250	soft 组织的 CA 服务，包含成员： peer1 、 admin1
`peer1.soft.ifantasy.net`	DebianA	7251	soft 组织的 peer1 成员节点
`web.ifantasy.net`	DebianB	7350	web 组织的 CA 服务，包含成员： peer1 、 admin1
`peer1.web.ifantasy.net`	DebianB	7351	web 组织的 peer1 成员节点
`hard.ifantasy.net`	DebianB	7450	hard 组织的 CA 服务，包含成员： peer1 、 admin1
`peer1.hard.ifantasy.net`	DebianB	7451	hard 组织的 peer1 成员节点

两个主机的相关信息为：

主机名	别名	网络地址	说明
DebianA	host1	172.25.1.250	运行 council 和 soft
DebianB	host2	172.25.1.251	运行 web 和 hard

实验准备

本文网络结构直接将 Hyperledger Fabric无排序组织以Raft协议启动多个Orderer服务、TLS组织运行维护Orderer服务中创建的 4-2_RunOrdererByCouncil 复制为 5_FabricNetworkByMultiHost 并修改（建议直接将本案例仓库 FabricLearn 下的 5_FabricNetworkByMultiHost 目录拷贝到本地运行），文中大部分命令在 Hyperledger Fabric定制联盟链网络工程实践中已有介绍因此不会详细说明。默认情况下，所有命令皆在 5_FabricNetworkByMultiHost 根目录下执行。

本系列所有实验都是在 VM ware 的 Debian 虚拟机（DebianA）下完成，本文会将 DebianA 虚拟机直接拷贝一份为 DebianB ，之后将会在 DebianA 下生成所有证书文件及通道文件，然后将文件复制一份到 DebianB 中再分别启动对应的网络。

配置文件

通过 docker 运行 fabric 网络总是需要解决不同节点间的通信问题（不能仅配置 DNS），目前主要有三种解决方案：

在 docker-compose.yaml 中设置 extra_hosts 字段
通过容器编排工具 docker swarm 实现
通过容器编排工具 Kubernetes(K8S) 实现（后期尝试）

大规模容器编排管理目前最流行的就是 K8S ，后期本人也会朝此方向尝试，为了简便本文使用第一种方式实现不同主机的 docker 容器通信。具体实现方面，只需要在 compose/docker-compose.yaml 中的 orderer 服务和 peer 服务中添加下列代码，如 orderer1.council.ifantasy.net ：

  orderer1.council.ifantasy.net:
    container_name: orderer1.council.ifantasy.net
    extends:
      file: docker-base.yaml
      service: orderer-base
    environment:
      - ORDERER_HOST=orderer1.council.ifantasy.net
      - ORDERER_GENERAL_LOCALMSPID=councilMSP
      - ORDERER_GENERAL_LISTENPORT=7051
    volumes:
      - ${LOCAL_CA_PATH}/council.ifantasy.net/registers/orderer1:${DOCKER_CA_PATH}/orderer
      - ${LOCAL_ROOT_PATH}/data/genesis.block:${DOCKER_CA_PATH}/orderer/genesis.block
    ports:
      - 7051:7051
      - 7052:8888
      - 7053:9999
    extra_hosts:
      - "orderer1.council.ifantasy.net:172.25.1.250"
      - "orderer2.council.ifantasy.net:172.25.1.250"
      - "orderer3.council.ifantasy.net:172.25.1.251"

如果不进行上述配置，则会因无法通信而出现下列错误：

Error: failed to send transaction: got unexpected status: SERVICE_UNAVAILABLE -- no Raft leader

证书和通道文件生成

网上很多相关教程都说明了将 Fabric 网络部署至多主机的方法^[1] ^[2]，大部分教程都是在同一台主机上生成全部的组织证书文件再进行证书分发部署（包括本文），但必须说明的是这种方式必然不能用于生产环境，因为生成组织证书的那台主机将会拥有全部组织的访问权限。在生产环境中，应该每个组织通过自身的 CA 服务生成自身的组织证书，并由单个组织创建通道后使用 Hyperledger Fabric组织的动态添加和删除中的方法将其它组织加入通道中。此外，毫无疑问使用 cryptogen 的方式一次性生成所有证书比本文所使用的 fabric-ca 的方式简单很多（不必考虑 DNS 问题）。

启动 CA 服务

由于要通过 DebainA 生成所有证书文件，所以得先将本地 DNS 指向 DebianA （setDNSTemp.sh）：

echo "127.0.0.1       council.ifantasy.net" >> /etc/hosts
echo "127.0.0.1       soft.ifantasy.net" >> /etc/hosts
echo "127.0.0.1       web.ifantasy.net" >> /etc/hosts
echo "127.0.0.1       hard.ifantasy.net" >> /etc/hosts

echo "127.0.0.1       orderer1.council.ifantasy.net" >> /etc/hosts
echo "127.0.0.1       orderer2.council.ifantasy.net" >> /etc/hosts
echo "127.0.0.1       orderer3.council.ifantasy.net" >> /etc/hosts

echo "127.0.0.1       peer1.soft.ifantasy.net" >> /etc/hosts
echo "127.0.0.1       peer1.web.ifantasy.net" >> /etc/hosts
echo "127.0.0.1       peer1.hard.ifantasy.net" >> /etc/hosts

直接运行根目录下的 0_Restart.sh 即可完成本实验所需 CA 服务的启动。

docker stop $(docker ps -aq)
docker rm $(docker ps -aq)
docker rmi $(docker images dev-* -q)
# rm -rf orgs data
docker-compose -f $LOCAL_ROOT_PATH/compose/docker-compose.yaml up -d council.ifantasy.net soft.ifantasy.net web.ifantasy.net hard.ifantasy.net

在前面的实验中，我们每次重启都删除所有的证书文件，但考虑到多机生成证书的复杂性，在这里只清除 docker 镜像而不删除证书文件。

注册账户

注册账户跟之前没什么不同，直接运行根目录下的 1_RegisterUser.sh 即可完成本实验所需用户的注册。

council 用户注册：

echo "Working on council"
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/council.ifantasy.net/ca/crypto/ca-cert.pem
export FABRIC_CA_CLIENT_HOME=$LOCAL_CA_PATH/council.ifantasy.net/ca/admin
fabric-ca-client enroll -d -u https://ca-admin:ca-adminpw@council.ifantasy.net:7050
fabric-ca-client register -d --id.name admin1 --id.secret admin1 --id.type admin -u https://council.ifantasy.net:7050
fabric-ca-client register -d --id.name orderer1 --id.secret orderer1 --id.type orderer -u https://council.ifantasy.net:7050
fabric-ca-client register -d --id.name orderer2 --id.secret orderer2 --id.type orderer -u https://council.ifantasy.net:7050
fabric-ca-client register -d --id.name orderer3 --id.secret orderer3 --id.type orderer -u https://council.ifantasy.net:7050
fabric-ca-client register -d --id.name peer1soft --id.secret peer1soft --id.type peer -u https://council.ifantasy.net:7050
fabric-ca-client register -d --id.name peer1web --id.secret peer1web --id.type peer -u https://council.ifantasy.net:7050
fabric-ca-client register -d --id.name peer1hard --id.secret peer1hard --id.type peer -u https://council.ifantasy.net:7050

soft 用户注册：

echo "Working on soft"
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/soft.ifantasy.net/ca/crypto/ca-cert.pem
export FABRIC_CA_CLIENT_HOME=$LOCAL_CA_PATH/soft.ifantasy.net/ca/admin
fabric-ca-client enroll -d -u https://ca-admin:ca-adminpw@soft.ifantasy.net:7250
fabric-ca-client register -d --id.name peer1 --id.secret peer1 --id.type peer -u https://soft.ifantasy.net:7250
fabric-ca-client register -d --id.name admin1 --id.secret admin1 --id.type admin -u https://soft.ifantasy.net:7250

web 用户注册：

echo "Working on web"
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/web.ifantasy.net/ca/crypto/ca-cert.pem
export FABRIC_CA_CLIENT_HOME=$LOCAL_CA_PATH/web.ifantasy.net/ca/admin
fabric-ca-client enroll -d -u https://ca-admin:ca-adminpw@web.ifantasy.net:7350
fabric-ca-client register -d --id.name peer1 --id.secret peer1 --id.type peer -u https://web.ifantasy.net:7350
fabric-ca-client register -d --id.name admin1 --id.secret admin1 --id.type admin -u https://web.ifantasy.net:7350

hard 用户注册：

echo "Working on hard"
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/hard.ifantasy.net/ca/crypto/ca-cert.pem
export FABRIC_CA_CLIENT_HOME=$LOCAL_CA_PATH/hard.ifantasy.net/ca/admin
fabric-ca-client enroll -d -u https://ca-admin:ca-adminpw@hard.ifantasy.net:7450
fabric-ca-client register -d --id.name peer1 --id.secret peer1 --id.type peer -u https://hard.ifantasy.net:7450
fabric-ca-client register -d --id.name admin1 --id.secret admin1 --id.type admin -u https://hard.ifantasy.net:7450

组织证书构建

组织证书构建跟之前的实验一样，直接运行根目录下的 2_EnrollUser.sh 即可完成本实验所需证书的构建。
直接运行根目录下的 2_EnrollUser.sh 即可完成本实验所需证书的构建。

组织资产预处理：

echo "Preparation============================="
mkdir -p $LOCAL_CA_PATH/council.ifantasy.net/assets
cp $LOCAL_CA_PATH/council.ifantasy.net/ca/crypto/ca-cert.pem $LOCAL_CA_PATH/council.ifantasy.net/assets/ca-cert.pem
cp $LOCAL_CA_PATH/council.ifantasy.net/ca/crypto/ca-cert.pem $LOCAL_CA_PATH/council.ifantasy.net/assets/tls-ca-cert.pem

mkdir -p $LOCAL_CA_PATH/soft.ifantasy.net/assets
cp $LOCAL_CA_PATH/soft.ifantasy.net/ca/crypto/ca-cert.pem $LOCAL_CA_PATH/soft.ifantasy.net/assets/ca-cert.pem
cp $LOCAL_CA_PATH/council.ifantasy.net/ca/crypto/ca-cert.pem $LOCAL_CA_PATH/soft.ifantasy.net/assets/tls-ca-cert.pem

mkdir -p $LOCAL_CA_PATH/web.ifantasy.net/assets 
cp $LOCAL_CA_PATH/web.ifantasy.net/ca/crypto/ca-cert.pem $LOCAL_CA_PATH/web.ifantasy.net/assets/ca-cert.pem
cp $LOCAL_CA_PATH/council.ifantasy.net/ca/crypto/ca-cert.pem $LOCAL_CA_PATH/web.ifantasy.net/assets/tls-ca-cert.pem

mkdir -p $LOCAL_CA_PATH/hard.ifantasy.net/assets
cp $LOCAL_CA_PATH/hard.ifantasy.net/ca/crypto/ca-cert.pem $LOCAL_CA_PATH/hard.ifantasy.net/assets/ca-cert.pem
cp $LOCAL_CA_PATH/council.ifantasy.net/ca/crypto/ca-cert.pem $LOCAL_CA_PATH/hard.ifantasy.net/assets/tls-ca-cert.pem
echo "Preparation end=========================="

council 证书构建：

echo "Start Council============================="
echo "Enroll Admin"
export FABRIC_CA_CLIENT_HOME=$LOCAL_CA_PATH/council.ifantasy.net/registers/admin1
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/council.ifantasy.net/assets/ca-cert.pem
export FABRIC_CA_CLIENT_MSPDIR=msp
fabric-ca-client enroll -d -u https://admin1:admin1@council.ifantasy.net:7050
# 加入通道时会用到admin/msp，其下必须要有admincers
mkdir -p $LOCAL_CA_PATH/council.ifantasy.net/registers/admin1/msp/admincerts
cp $LOCAL_CA_PATH/council.ifantasy.net/registers/admin1/msp/signcerts/cert.pem $LOCAL_CA_PATH/council.ifantasy.net/registers/admin1/msp/admincerts/cert.pem

echo "Enroll Orderer1"
# for identity
export FABRIC_CA_CLIENT_HOME=$LOCAL_CA_PATH/council.ifantasy.net/registers/orderer1
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/council.ifantasy.net/assets/ca-cert.pem
export FABRIC_CA_CLIENT_MSPDIR=msp
fabric-ca-client enroll -d -u https://orderer1:orderer1@council.ifantasy.net:7050
mkdir -p $LOCAL_CA_PATH/council.ifantasy.net/registers/orderer1/msp/admincerts
cp $LOCAL_CA_PATH/council.ifantasy.net/registers/admin1/msp/signcerts/cert.pem $LOCAL_CA_PATH/council.ifantasy.net/registers/orderer1/msp/admincerts/cert.pem
# for TLS
export FABRIC_CA_CLIENT_MSPDIR=tls-msp
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/council.ifantasy.net/assets/tls-ca-cert.pem
fabric-ca-client enroll -d -u https://orderer1:orderer1@council.ifantasy.net:7050 --enrollment.profile tls --csr.hosts orderer1.council.ifantasy.net
cp $LOCAL_CA_PATH/council.ifantasy.net/registers/orderer1/tls-msp/keystore/*_sk $LOCAL_CA_PATH/council.ifantasy.net/registers/orderer1/tls-msp/keystore/key.pem

echo "Enroll Orderer2"
# for identity
export FABRIC_CA_CLIENT_HOME=$LOCAL_CA_PATH/council.ifantasy.net/registers/orderer2
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/council.ifantasy.net/assets/ca-cert.pem
export FABRIC_CA_CLIENT_MSPDIR=msp
fabric-ca-client enroll -d -u https://orderer2:orderer2@council.ifantasy.net:7050
mkdir -p $LOCAL_CA_PATH/council.ifantasy.net/registers/orderer2/msp/admincerts
cp $LOCAL_CA_PATH/council.ifantasy.net/registers/admin1/msp/signcerts/cert.pem $LOCAL_CA_PATH/council.ifantasy.net/registers/orderer2/msp/admincerts/cert.pem
# for TLS
export FABRIC_CA_CLIENT_MSPDIR=tls-msp
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/council.ifantasy.net/assets/tls-ca-cert.pem
fabric-ca-client enroll -d -u https://orderer2:orderer2@council.ifantasy.net:7050 --enrollment.profile tls --csr.hosts orderer2.council.ifantasy.net
cp $LOCAL_CA_PATH/council.ifantasy.net/registers/orderer2/tls-msp/keystore/*_sk $LOCAL_CA_PATH/council.ifantasy.net/registers/orderer2/tls-msp/keystore/key.pem

echo "Enroll Orderer3"
# for identity
export FABRIC_CA_CLIENT_HOME=$LOCAL_CA_PATH/council.ifantasy.net/registers/orderer3
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/council.ifantasy.net/assets/ca-cert.pem
export FABRIC_CA_CLIENT_MSPDIR=msp
fabric-ca-client enroll -d -u https://orderer3:orderer3@council.ifantasy.net:7050
mkdir -p $LOCAL_CA_PATH/council.ifantasy.net/registers/orderer3/msp/admincerts
cp $LOCAL_CA_PATH/council.ifantasy.net/registers/admin1/msp/signcerts/cert.pem $LOCAL_CA_PATH/council.ifantasy.net/registers/orderer3/msp/admincerts/cert.pem
# for TLS
export FABRIC_CA_CLIENT_MSPDIR=tls-msp
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/council.ifantasy.net/assets/tls-ca-cert.pem
fabric-ca-client enroll -d -u https://orderer3:orderer3@council.ifantasy.net:7050 --enrollment.profile tls --csr.hosts orderer3.council.ifantasy.net
cp $LOCAL_CA_PATH/council.ifantasy.net/registers/orderer3/tls-msp/keystore/*_sk $LOCAL_CA_PATH/council.ifantasy.net/registers/orderer3/tls-msp/keystore/key.pem

mkdir -p $LOCAL_CA_PATH/council.ifantasy.net/msp/admincerts
mkdir -p $LOCAL_CA_PATH/council.ifantasy.net/msp/cacerts
mkdir -p $LOCAL_CA_PATH/council.ifantasy.net/msp/tlscacerts
mkdir -p $LOCAL_CA_PATH/council.ifantasy.net/msp/users
cp $LOCAL_CA_PATH/council.ifantasy.net/assets/ca-cert.pem $LOCAL_CA_PATH/council.ifantasy.net/msp/cacerts/
cp $LOCAL_CA_PATH/council.ifantasy.net/assets/tls-ca-cert.pem $LOCAL_CA_PATH/council.ifantasy.net/msp/tlscacerts/
cp $LOCAL_CA_PATH/council.ifantasy.net/registers/admin1/msp/signcerts/cert.pem $LOCAL_CA_PATH/council.ifantasy.net/msp/admincerts/cert.pem
cp $LOCAL_ROOT_PATH/config/config-msp.yaml $LOCAL_CA_PATH/council.ifantasy.net/msp/config.yaml
echo "End council============================="

soft 证书构建：

echo "Start Soft============================="
echo "Enroll Admin"
export FABRIC_CA_CLIENT_HOME=$LOCAL_CA_PATH/soft.ifantasy.net/registers/admin1
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/soft.ifantasy.net/assets/ca-cert.pem
export FABRIC_CA_CLIENT_MSPDIR=msp
fabric-ca-client enroll -d -u https://admin1:admin1@soft.ifantasy.net:7250
mkdir -p $LOCAL_CA_PATH/soft.ifantasy.net/registers/admin1/msp/admincerts
cp $LOCAL_CA_PATH/soft.ifantasy.net/registers/admin1/msp/signcerts/cert.pem $LOCAL_CA_PATH/soft.ifantasy.net/registers/admin1/msp/admincerts/cert.pem

echo "Enroll Peer1"
export FABRIC_CA_CLIENT_HOME=$LOCAL_CA_PATH/soft.ifantasy.net/registers/peer1
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/soft.ifantasy.net/assets/ca-cert.pem
export FABRIC_CA_CLIENT_MSPDIR=msp
fabric-ca-client enroll -d -u https://peer1:peer1@soft.ifantasy.net:7250
# for TLS
export FABRIC_CA_CLIENT_MSPDIR=tls-msp
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/soft.ifantasy.net/assets/tls-ca-cert.pem
fabric-ca-client enroll -d -u https://peer1soft:peer1soft@council.ifantasy.net:7050 --enrollment.profile tls --csr.hosts peer1.soft.ifantasy.net
cp $LOCAL_CA_PATH/soft.ifantasy.net/registers/peer1/tls-msp/keystore/*_sk $LOCAL_CA_PATH/soft.ifantasy.net/registers/peer1/tls-msp/keystore/key.pem
mkdir -p $LOCAL_CA_PATH/soft.ifantasy.net/registers/peer1/msp/admincerts
cp $LOCAL_CA_PATH/soft.ifantasy.net/registers/admin1/msp/signcerts/cert.pem $LOCAL_CA_PATH/soft.ifantasy.net/registers/peer1/msp/admincerts/cert.pem

mkdir -p $LOCAL_CA_PATH/soft.ifantasy.net/msp/admincerts
mkdir -p $LOCAL_CA_PATH/soft.ifantasy.net/msp/cacerts
mkdir -p $LOCAL_CA_PATH/soft.ifantasy.net/msp/tlscacerts
mkdir -p $LOCAL_CA_PATH/soft.ifantasy.net/msp/users
cp $LOCAL_CA_PATH/soft.ifantasy.net/assets/ca-cert.pem $LOCAL_CA_PATH/soft.ifantasy.net/msp/cacerts/
cp $LOCAL_CA_PATH/soft.ifantasy.net/assets/tls-ca-cert.pem $LOCAL_CA_PATH/soft.ifantasy.net/msp/tlscacerts/
cp $LOCAL_CA_PATH/soft.ifantasy.net/registers/admin1/msp/signcerts/cert.pem $LOCAL_CA_PATH/soft.ifantasy.net/msp/admincerts/cert.pem
cp $LOCAL_ROOT_PATH/config/config-msp.yaml $LOCAL_CA_PATH/soft.ifantasy.net/msp/config.yaml
echo "End Soft============================="

web 证书构建：

echo "Start Web============================="
echo "Enroll Admin"
export FABRIC_CA_CLIENT_HOME=$LOCAL_CA_PATH/web.ifantasy.net/registers/admin1
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/web.ifantasy.net/assets/ca-cert.pem
export FABRIC_CA_CLIENT_MSPDIR=msp
fabric-ca-client enroll -d -u https://admin1:admin1@web.ifantasy.net:7350
mkdir -p $LOCAL_CA_PATH/web.ifantasy.net/registers/admin1/msp/admincerts
cp $LOCAL_CA_PATH/web.ifantasy.net/registers/admin1/msp/signcerts/cert.pem $LOCAL_CA_PATH/web.ifantasy.net/registers/admin1/msp/admincerts/cert.pem

echo "Enroll Peer1"
# for identity
export FABRIC_CA_CLIENT_HOME=$LOCAL_CA_PATH/web.ifantasy.net/registers/peer1
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/web.ifantasy.net/assets/ca-cert.pem
export FABRIC_CA_CLIENT_MSPDIR=msp
fabric-ca-client enroll -d -u https://peer1:peer1@web.ifantasy.net:7350
# for TLS
export FABRIC_CA_CLIENT_MSPDIR=tls-msp
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/web.ifantasy.net/assets/tls-ca-cert.pem
fabric-ca-client enroll -d -u https://peer1web:peer1web@council.ifantasy.net:7050 --enrollment.profile tls --csr.hosts peer1.web.ifantasy.net
cp $LOCAL_CA_PATH/web.ifantasy.net/registers/peer1/tls-msp/keystore/*_sk $LOCAL_CA_PATH/web.ifantasy.net/registers/peer1/tls-msp/keystore/key.pem
mkdir -p $LOCAL_CA_PATH/web.ifantasy.net/registers/peer1/msp/admincerts
cp $LOCAL_CA_PATH/web.ifantasy.net/registers/admin1/msp/signcerts/cert.pem $LOCAL_CA_PATH/web.ifantasy.net/registers/peer1/msp/admincerts/cert.pem

mkdir -p $LOCAL_CA_PATH/web.ifantasy.net/msp/admincerts
mkdir -p $LOCAL_CA_PATH/web.ifantasy.net/msp/cacerts
mkdir -p $LOCAL_CA_PATH/web.ifantasy.net/msp/tlscacerts
mkdir -p $LOCAL_CA_PATH/web.ifantasy.net/msp/users
cp $LOCAL_CA_PATH/web.ifantasy.net/assets/ca-cert.pem $LOCAL_CA_PATH/web.ifantasy.net/msp/cacerts/
cp $LOCAL_CA_PATH/web.ifantasy.net/assets/tls-ca-cert.pem $LOCAL_CA_PATH/web.ifantasy.net/msp/tlscacerts/
cp $LOCAL_CA_PATH/web.ifantasy.net/registers/admin1/msp/signcerts/cert.pem $LOCAL_CA_PATH/web.ifantasy.net/msp/admincerts/cert.pem
cp $LOCAL_ROOT_PATH/config/config-msp.yaml $LOCAL_CA_PATH/web.ifantasy.net/msp/config.yaml
echo "End Web============================="

hard 证书构建：

echo "Start Hard============================="
echo "Enroll Admin"
export FABRIC_CA_CLIENT_HOME=$LOCAL_CA_PATH/hard.ifantasy.net/registers/admin1
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/hard.ifantasy.net/assets/ca-cert.pem
export FABRIC_CA_CLIENT_MSPDIR=msp
fabric-ca-client enroll -d -u https://admin1:admin1@hard.ifantasy.net:7450
mkdir -p $LOCAL_CA_PATH/hard.ifantasy.net/registers/admin1/msp/admincerts
cp $LOCAL_CA_PATH/hard.ifantasy.net/registers/admin1/msp/signcerts/cert.pem $LOCAL_CA_PATH/hard.ifantasy.net/registers/admin1/msp/admincerts/cert.pem

echo "Enroll Peer1"
export FABRIC_CA_CLIENT_HOME=$LOCAL_CA_PATH/hard.ifantasy.net/registers/peer1
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/hard.ifantasy.net/assets/ca-cert.pem
export FABRIC_CA_CLIENT_MSPDIR=msp
fabric-ca-client enroll -d -u https://peer1:peer1@hard.ifantasy.net:7450
# for TLS
export FABRIC_CA_CLIENT_MSPDIR=tls-msp
export FABRIC_CA_CLIENT_TLS_CERTFILES=$LOCAL_CA_PATH/hard.ifantasy.net/assets/tls-ca-cert.pem
fabric-ca-client enroll -d -u https://peer1hard:peer1hard@council.ifantasy.net:7050 --enrollment.profile tls --csr.hosts peer1.hard.ifantasy.net
cp $LOCAL_CA_PATH/hard.ifantasy.net/registers/peer1/tls-msp/keystore/*_sk $LOCAL_CA_PATH/hard.ifantasy.net/registers/peer1/tls-msp/keystore/key.pem
mkdir -p $LOCAL_CA_PATH/hard.ifantasy.net/registers/peer1/msp/admincerts
cp $LOCAL_CA_PATH/hard.ifantasy.net/registers/admin1/msp/signcerts/cert.pem $LOCAL_CA_PATH/hard.ifantasy.net/registers/peer1/msp/admincerts/cert.pem

mkdir -p $LOCAL_CA_PATH/hard.ifantasy.net/msp/admincerts
mkdir -p $LOCAL_CA_PATH/hard.ifantasy.net/msp/cacerts
mkdir -p $LOCAL_CA_PATH/hard.ifantasy.net/msp/tlscacerts
mkdir -p $LOCAL_CA_PATH/hard.ifantasy.net/msp/users
cp $LOCAL_CA_PATH/hard.ifantasy.net/assets/ca-cert.pem $LOCAL_CA_PATH/hard.ifantasy.net/msp/cacerts/
cp $LOCAL_CA_PATH/hard.ifantasy.net/assets/tls-ca-cert.pem $LOCAL_CA_PATH/hard.ifantasy.net/msp/tlscacerts/
cp $LOCAL_CA_PATH/hard.ifantasy.net/registers/admin1/msp/signcerts/cert.pem $LOCAL_CA_PATH/hard.ifantasy.net/msp/admincerts/cert.pem
cp $LOCAL_ROOT_PATH/config/config-msp.yaml $LOCAL_CA_PATH/hard.ifantasy.net/msp/config.yaml
echo "End Hard============================="

在上面操作完成后，已经暂时不需要 CA 服务了，因此先使用 docker stop $(docker ps -aq) 命令关闭正在运行的四个 CA 容器。

配置通道

配置通道的方法跟单机略有区别，由于我们预期将 peer 和 orderer 服务部署在不同的主机上，因此并不需要使用 docker-compose 启动其它容器，只需要生成通道文件就好。运行根目录下的 3_Configtxgen.sh 即可完成本实验所需通道配置。

configtxgen -profile OrgsChannel -outputCreateChannelTx $LOCAL_ROOT_PATH/data/testchannel.tx -channelID testchannel
configtxgen -profile OrgsChannel -outputBlock $LOCAL_ROOT_PATH/data/testchannel.block -channelID testchannel

cp $LOCAL_ROOT_PATH/data/testchannel.block $LOCAL_CA_PATH/soft.ifantasy.net/assets/
cp $LOCAL_ROOT_PATH/data/testchannel.block $LOCAL_CA_PATH/web.ifantasy.net/assets/
cp $LOCAL_ROOT_PATH/data/testchannel.block $LOCAL_CA_PATH/hard.ifantasy.net/assets/

在以上步骤完成后，在 5_FabricNetworkByMultiHost 文件夹下的 data 和 orgs 目录中已经生成了全部网络所需的通道文件和组织证书文件，现在我们将 5_FabricNetworkByMultiHost 文件夹复制一份到 DebianB 主机上开始接下来的实验。以后每次重启网络只需要在每个主机上运行 0_Restart.sh 、 4_JoinChannel_host1.sh 、 4_JoinChannel_host2.sh 、 5_TestChaincode_host1.sh 、 5_TestChaincode_host2.sh 。

启动多机网络

配置 DNS

在上节中，我们为了方便在 DebianA 上生成证书，将所有域名映射都指向了 DebianA 自身，现在需要手动修改 /etc/hosts 文件并删除上节设置的 DNS 映射，然后设置新的 DNS 内容：

echo "172.25.1.250       council.ifantasy.net" >> /etc/hosts
echo "172.25.1.250       soft.ifantasy.net" >> /etc/hosts
echo "172.25.1.251       web.ifantasy.net" >> /etc/hosts
echo "172.25.1.251       hard.ifantasy.net" >> /etc/hosts

echo "172.25.1.250       orderer1.council.ifantasy.net" >> /etc/hosts
echo "172.25.1.250       orderer2.council.ifantasy.net" >> /etc/hosts
echo "172.25.1.251       orderer3.council.ifantasy.net" >> /etc/hosts

echo "172.25.1.250       peer1.soft.ifantasy.net" >> /etc/hosts
echo "172.25.1.251       peer1.web.ifantasy.net" >> /etc/hosts
echo "172.25.1.251       peer1.hard.ifantasy.net" >> /etc/hosts

同样，我们需要在 DebianB 上设置类似的 DNS 映射：

echo "172.25.1.250       council.ifantasy.net" >> /etc/hosts
echo "172.25.1.250       soft.ifantasy.net" >> /etc/hosts
echo "172.25.1.251       web.ifantasy.net" >> /etc/hosts
echo "172.25.1.251       hard.ifantasy.net" >> /etc/hosts

echo "172.25.1.250       orderer1.council.ifantasy.net" >> /etc/hosts
echo "172.25.1.250       orderer2.council.ifantasy.net" >> /etc/hosts
echo "172.25.1.251       orderer3.council.ifantasy.net" >> /etc/hosts

echo "172.25.1.250       peer1.soft.ifantasy.net" >> /etc/hosts
echo "172.25.1.251       peer1.web.ifantasy.net" >> /etc/hosts
echo "172.25.1.251       peer1.hard.ifantasy.net" >> /etc/hosts

启动容器并加入通道

DebainA

可以直接运行根目录下的 4_JoinChannel_host1.sh 脚本以使 DebianA 执行下列命令启动容器并加入通道：

启动本主机容器：

source envpeer1soft
docker-compose -f $LOCAL_ROOT_PATH/compose/docker-compose.yaml up -d council.ifantasy.net soft.ifantasy.net peer1.soft.ifantasy.net 
docker-compose -f $LOCAL_ROOT_PATH/compose/docker-compose.yaml up -d orderer1.council.ifantasy.net orderer2.council.ifantasy.net

此时 DebianA 运行的容器网络为：
DebianA 容器网络
2. 本主机排序服务加入通道：

source envpeer1soft
export ORDERER_ADMIN_TLS_SIGN_CERT=$LOCAL_CA_PATH/council.ifantasy.net/registers/orderer1/tls-msp/signcerts/cert.pem
export ORDERER_ADMIN_TLS_PRIVATE_KEY=$LOCAL_CA_PATH/council.ifantasy.net/registers/orderer1/tls-msp/keystore/key.pem
osnadmin channel join -o orderer1.council.ifantasy.net:7052 --channelID testchannel --config-block $LOCAL_ROOT_PATH/data/testchannel.block --ca-file "$ORDERER_CA" --client-cert "$ORDERER_ADMIN_TLS_SIGN_CERT" --client-key "$ORDERER_ADMIN_TLS_PRIVATE_KEY"
osnadmin channel list -o orderer1.council.ifantasy.net:7052 --ca-file $ORDERER_CA --client-cert $ORDERER_ADMIN_TLS_SIGN_CERT --client-key $ORDERER_ADMIN_TLS_PRIVATE_KEY
export ORDERER_ADMIN_TLS_SIGN_CERT=$LOCAL_CA_PATH/council.ifantasy.net/registers/orderer2/tls-msp/signcerts/cert.pem
export ORDERER_ADMIN_TLS_PRIVATE_KEY=$LOCAL_CA_PATH/council.ifantasy.net/registers/orderer2/tls-msp/keystore/key.pem
osnadmin channel join -o orderer2.council.ifantasy.net:7055 --channelID testchannel --config-block $LOCAL_ROOT_PATH/data/testchannel.block --ca-file "$ORDERER_CA" --client-cert "$ORDERER_ADMIN_TLS_SIGN_CERT" --client-key "$ORDERER_ADMIN_TLS_PRIVATE_KEY"
osnadmin channel list -o orderer2.council.ifantasy.net:7055 --ca-file $ORDERER_CA --client-cert $ORDERER_ADMIN_TLS_SIGN_CERT --client-key $ORDERER_ADMIN_TLS_PRIVATE_KEY

本主机组织加入通道：

source envpeer1soft
peer channel join -b $LOCAL_CA_PATH/soft.ifantasy.net/assets/testchannel.block
peer channel list

DebianB

可以直接运行根目录下的 4_JoinChannel_host2.sh 脚本以使 DebianB 执行下列命令启动容器并加入通道：

启动本主机容器：

source envpeer1web
docker-compose -f $LOCAL_ROOT_PATH/compose/docker-compose.yaml up -d web.ifantasy.net peer1.web.ifantasy.net hard.ifantasy.net peer1.hard.ifantasy.net 
docker-compose -f $LOCAL_ROOT_PATH/compose/docker-compose.yaml up -d orderer3.council.ifantasy.net

此时 DebianB 运行的容器网络为：
DebianB 容器网络
2. 本主机排序服务加入通道：

source envpeer1web
export ORDERER_ADMIN_TLS_SIGN_CERT=$LOCAL_CA_PATH/council.ifantasy.net/registers/orderer3/tls-msp/signcerts/cert.pem
export ORDERER_ADMIN_TLS_PRIVATE_KEY=$LOCAL_CA_PATH/council.ifantasy.net/registers/orderer3/tls-msp/keystore/key.pem
osnadmin channel join -o orderer3.council.ifantasy.net:7058 --channelID testchannel --config-block $LOCAL_ROOT_PATH/data/testchannel.block --ca-file "$ORDERER_CA" --client-cert "$ORDERER_ADMIN_TLS_SIGN_CERT" --client-key "$ORDERER_ADMIN_TLS_PRIVATE_KEY"
osnadmin channel list -o orderer3.council.ifantasy.net:7058 --ca-file $ORDERER_CA --client-cert $ORDERER_ADMIN_TLS_SIGN_CERT --client-key $ORDERER_ADMIN_TLS_PRIVATE_KEY

本主机组织加入通道：

source envpeer1web
peer channel join -b $LOCAL_CA_PATH/web.ifantasy.net/assets/testchannel.block
peer channel list
source envpeer1hard
peer channel join -b $LOCAL_CA_PATH/hard.ifantasy.net/assets/testchannel.block
peer channel list

安装并测试链码

由于通道更新需要根据策略进行顺序操作，所以不可以直接运行根目录下的 5_TestChaincode_host1.sh 脚本，而是在不同主机中分别按链码周期运行对应的脚本内容：

DebianA 安装链码：

source envpeer1soft
# peer lifecycle chaincode package basic.tar.gz --path asset-transfer-basic/chaincode-go --label basic_1
peer lifecycle chaincode install basic.tar.gz
peer lifecycle chaincode queryinstalled

DebianB 安装链码：

source envpeer1web
peer lifecycle chaincode install basic.tar.gz
peer lifecycle chaincode queryinstalled
source envpeer1hard
peer lifecycle chaincode install basic.tar.gz
peer lifecycle chaincode queryinstalled

DebianA 批准链码：

export CHAINCODE_ID=basic_1:06613e463ef6694805dd896ca79634a2de36fdf019fa7976467e6e632104d718
source envpeer1soft
peer lifecycle chaincode approveformyorg -o orderer1.council.ifantasy.net:7051 --tls --cafile $ORDERER_CA  --channelID testchannel --name basic --version 1.0 --sequence 1 --waitForEvent --init-required --package-id $CHAINCODE_ID
peer lifecycle chaincode queryapproved -C testchannel -n basic --sequence 1

此时使用以下命令查看链码批准情况：

peer lifecycle chaincode checkcommitreadiness -o orderer1.council.ifantasy.net:7051 --tls --cafile $ORDERER_CA --channelID testchannel --name basic --version 1.0 --sequence 1 --init-required

DebianA 链码批准情况1
4. DebainB 批准链码：

export CHAINCODE_ID=basic_1:06613e463ef6694805dd896ca79634a2de36fdf019fa7976467e6e632104d718
source envpeer1web
peer lifecycle chaincode approveformyorg -o orderer1.council.ifantasy.net:7051 --tls --cafile $ORDERER_CA  --channelID testchannel --name basic --version 1.0 --sequence 1 --waitForEvent --init-required --package-id $CHAINCODE_ID
peer lifecycle chaincode queryapproved -C testchannel -n basic --sequence 1
source envpeer1hard
peer lifecycle chaincode approveformyorg -o orderer1.council.ifantasy.net:7051 --tls --cafile $ORDERER_CA  --channelID testchannel --name basic --version 1.0 --sequence 1 --waitForEvent --init-required --package-id $CHAINCODE_ID
peer lifecycle chaincode queryapproved -C testchannel -n basic --sequence 1

此时再回到 DebianA 查看链码批准情况发现已同步：
DebianA 链码批准情况2
5. DebainB 提交链码：

source envpeer1web
peer lifecycle chaincode commit -o orderer1.council.ifantasy.net:7051 --tls --cafile $ORDERER_CA --channelID testchannel --name basic --init-required --version 1.0 --sequence 1 --peerAddresses peer1.soft.ifantasy.net:7251 --tlsRootCertFiles $CORE_PEER_TLS_ROOTCERT_FILE --peerAddresses peer1.web.ifantasy.net:7351 --tlsRootCertFiles $CORE_PEER_TLS_ROOTCERT_FILE

DebainB 初始化链码：

source envpeer1web
peer chaincode invoke --isInit -o orderer1.council.ifantasy.net:7051 --tls --cafile $ORDERER_CA --channelID testchannel --name basic --peerAddresses peer1.soft.ifantasy.net:7251 --tlsRootCertFiles $CORE_PEER_TLS_ROOTCERT_FILE --peerAddresses peer1.web.ifantasy.net:7351 --tlsRootCertFiles $CORE_PEER_TLS_ROOTCERT_FILE -c '{"Args":["InitLedger"]}'

DebainA 调用链码：

peer chaincode invoke -o orderer1.council.ifantasy.net:7051 --tls --cafile $ORDERER_CA --channelID testchannel --name basic --peerAddresses peer1.soft.ifantasy.net:7251 --tlsRootCertFiles $CORE_PEER_TLS_ROOTCERT_FILE --peerAddresses peer1.web.ifantasy.net:7351 --tlsRootCertFiles $CORE_PEER_TLS_ROOTCERT_FILE -c '{"Args":["GetAllAssets"]}'

DebainA 调用链码：

参考

KC Tam. Multi-Host Deployment for First Network (Hyperledger Fabric v2). CSDN. [2020-08-11] ↩︎
余府. Hyperledger Fabric 2.x 多机部署/分布式集群部署流程. CSDN. [2020-11-28] ↩︎

原创文章，作者：kirin，如若转载，请注明出处：https://blog.ytso.com/244360.html