docker搭建mongodb高可用集群

构建mongodb集群的docker

参考资料：

• 基于 Docker 的 MongoDB 主从集群

前序–谈谈数据库升级方案

• 一主一从
• 一主两从
• 一主一从一仲裁 (当前搭建使用)
• 集群shard分片(暂不讨论)

1.使用docker构建相应的单节点mongodb服务

#masterdocker主节点 run --name mongo-master --restart=always -d --net="bridge" -p 27017:27017 \-v /root/fct/mongocluster/master/data/db:/data/db \-v /root/fct/mongocluster/master/logs/mongodb:/var/log/mongodb \mongo:3.4 \/bin/sh -c 'mongod --dbpath /data/db --replSet annosys'

#slave1docker run --name mongo-slave1 --restart=always -d --net="bridge" -p 27027:27017  \-v /root/fct/mongocluster/slave1/////data/db:/data/db \-v /root/fct/mongocluster/slave1/////logs/mongodb:/var/log/mongodb \mongo:3.4 \/bin/sh -c 'mongod --dbpath /data/db --replSet annosys'

#仲裁节点arbiterdock run --name mongo-arbiter -d  --net="bridge" -p 27037:27017 \mongo:3.4 \/bin/sh -c 'mongod --dbpath /data/db --replSet annosys --smallfiles'

通过配置构建每个单个mongdb，以构建一个mongodb集群

2 副本集的初始配置

2.1三台机器中的任何一台登录mongodb

进入任何mongodb容器，然后

mongo

切换到admin 库

use admin

在这里定义配置变量的副本集 _id:” annosys” 与上述配置文件中replset=anosys相同。

config={ _id:"annosys", members:[{_id:0,host:'172.19.32.142:27017',priority:5},{_id:1,host:'172.19.32.142:27027',priority:3},{_id:2,host:'172.19.32.142:27037',arbiterOnly:true}]}

副本集配置的初始化

rs.initiate(config)

检查集群节点的状态rs.status()

annosys:SECONDARY> rs.status(){"set" : "annosys","date" : ISODate("2019-10-23T09:43:42.784Z)，myState" : 2,"term" : NumberLong(1),"syncingTo" : "172.19.32.142:27017","syncSourceHost" : "172.19.32.142:27017","syncSourceId" : 0,"heartbeatIntervalMillis" : NumberLong(2000),"optimes" : {"lastCommittedOpTime" : {"ts" : Timestamp(1571823820, 1),"t" : NumberLong(1)},"appliedOpTime" : {"ts" : Timestamp(1571823820, 1),"t" : NumberLong(1)},"durableOpTime" : {"ts" : Timestamp(1571823820, 1),"t" : NumberLong(1)}members" : [{"_id" : 0,"name" : "172.19.32.142:27017","health" : 1,"state" : 1,"stateStr" : "PRIMARY","uptime" : 3032,"optime" : {"ts" : Timestamp(1571823820, 1),"t" : NumberLong(1)},"optimeDurable" : {"ts" : Timestamp(1571823820, 1),"t" : NumberLong(1)},"optimeDate" : ISODate("2019-10-23T09:43:40Z"),"optimeDurableDate" : ISODate("2019-10-23T09:43:40Z"),"lastHeartbeat" : ISODate("2019-10-23T09:43:42.362Z"),"lastHeartbeatRecv" : ISODate("2019-10-23T09:43:42.036Z)，pingMs" : NumberLong(0),"lastHeartbeatMessage" : "","syncingTo" : "","syncSourceHost" : "","syncSourceId" : -1,"infoMessage" : "","electionTime" : Timestamp(1571820799, 1),"electionDate" : ISODate("2019-10-23T08:53:19Z"),"configVersion" : 1},{"_id" : 1,"name" : "172.19.32.142:27027","health" : 1,"state" : 2,"stateStr" : "SECONDARY","uptime" : 4498,"optime" : {"ts" : Timestamp(1571823820, 1),"t" : NumberLong(1)},"optimeDate" : ISODate("2019-10-23T09:43:40Z"),"syncingTo" : "172.19.32.142:27017","syncSourceHost" : "172.19.32.142:27017","syncSourceId" : 0,"infoMessage" : "","configVersion" : 1,"self" : true,"lastHeartbeatMessage" : ""},{"_id" : 2,"name" : "172.19.32.142:27037","health" : 1,"state" : 7,"stateStr" : "ARBITER","uptime" : 3032,"lastHeartbeat" : ISODate("2019-10-23T09:43:42.362Z)，lastHeartbeatRecv" : ISODate("2019-10-23T09:43:(40.850Z)，pingMs" : NumberLong(0),"lastHeartbeatMessage" : "","syncingTo" : "","syncSourceHost" : "","syncSourceId" : -1,"infoMessage" : "","configVersion" : 1}],"ok" : 1}

各节点健康状况正常。

3 节点数据同步检查测试、读写分离测试、故障测试模拟3.1数据同步测试

如检查过程中出现以下错误：not master and slaveOk=false

annosys:SECONDARY> use test;switched to db testannosys:SECONDARY> show tables;2019-10-23T09:44:28.838+0000 E QUERY    [thread1] Error: listCollections failed: {"ok" : 0,"errmsg" : "not master and slaveOk=false","code" : 13435,"codeName" : "NotMasterNoSlaveOk"}：_getErrorWithCode@src/mongo/shell/utils.js:25:13DB.prototype._getCollectionInfosCommand@src/mongo/shell/db.js:807:1DB.prototype.getCollectionInfos@src/mongo/shell/db.js:819:19DB.prototype.getCollectionNames@src/mongo/shell/db.js:830:Helper16shell.show@src/mongo/shell/utils.js:807:Helper9shell@src/mongo/shell/utils.js:704:15

此时需要执行rs.slaveOk()命令即可

annosys:SECONDARY> rs.slaveOk()annosys:SECONDARY> use testswitched to db testannosys:SECONDARY> db.testdb.find(){ "_id" : ObjectId(5db01956f6d1094b8e5c0a16) “test1” : testval1 }{ "_id" : ObjectId(5db01f167be4feac94c9f596) "name" : "fct----1" }{ "_id" : ObjectId(5db01f167be4feac94c9f597) "name" : "fct----2" }{ "_id" : ObjectId(5db01f167be4feac94cf599) "name" : "fct----4" }{ "_id" : ObjectId(5db01f167be4feac94c9f598) "name" : "fct----3" }

若发现数据同步，则证明设置正常。

三、二读写分离检查

目前的集群是，只能在master中写数据，然后在salve中读数据，

如果在slave中执行写数据，请报告以下错误

annosys:SECONDARY> db.testdb.insert({msg: 'this is from primary change yyy', ts: new Date()})WriteResult({ "writeError" : { "code" : 10107, "errmsg" : "not master" } })

master节点没问题

annosys:PRIMARY> db.testdb.insert({msg: 'this is from primary change xxx', ts: new Date()})WriteResult({ "nInserted" : 1 })

说明节点读写分离。

3.3 模拟故障测试

当docker停止master主节点服务时，在slave节点容器中执行插入动作时，

annosys:SECONDARY> db.testdb.insert({msg: 'this is from primary change yyy', ts: new Date()})WriteResult({ "writeError" : { "code" : 10107, "errmsg" : "not master" } })annosys:SECONDARY> db.testdb.insert({msg: 'this is from primary change yyy', ts: new Date()})WriteResult({ "nInserted" : 1 })annosys:PRIMARY> db.testdb.insert({msg: 'this is from primary change yyy', ts: new Date()})WriteResult({ "nInserted" : 1 })

结果表明：

①此时只能从节点slave1读取的节点，也可以写数据；

②注意查看原来的annosys:SECONDARY从节点升级到annosys:PRIMARY主节点了。

说明故障转移已成功完成。

到目前为止，测试mongodb集群建设和高可用性测试已经完成！

(如果后期需要从节点添加，可以直接修改配置文件，更新配置！使用时需要测试)

4 Java连接测试和客户端可视化界面4.1如果需要使用客户端，建议使用Robot 3T连接mongodb集群。

4.2 Java连接mongodb集群测试

package mongotest;import com.mongodb.MongoClient;import com.mongodb.ServerAddress;import com.mongodb.client.FindIterable;import com.mongodb.client.MongoCollection;import com.mongodb.client.MongoDatabase;import org.bson.Document;import java.util.ArrayList;import java.util.List;/** * 描述: java连接mongdb集群；测试高可用性 * @author: fangchangtan * @version 创建时间：2018年11月26日 下午7:45:29 */public class TestMongoDBReplSet {    public static void main(String[] args) {        ArrayList<ServerAddress> arrayList = new ArrayList<>();        try {            java连接//mongpdb配置为集群模式            List<ServerAddress> addresses = new ArrayList<ServerAddress>();            ServerAddress address1 = new ServerAddress(172.19.32.142 , 27017);            ServerAddress addres2 = new ServerAddress(172.19.32.142 , 27027);            ServerAddress address3 = new ServerAddress(172.19.32.142 , 27037);            addresses.add(address1);            addresses.add(addres2);            addresses.add(addres3);            MongoClient mongoClient = new MongoClient(addresses);            MongoDatabase database = mongoClient.getDatabase("test");            MongoCollection<Document> collection = database.getCollection("testdb");            for (int i = 0; i < 1000; i++) {                System.out.println=========================;                Thread.sleep(2000);                /////将文档数据插入集群                collection.insertOne(new Document("name","dog"+i));                ////查询集群中的数据记录                FindIterable<Document> find = collection.find();                for(Document doc : find){                    System.out.println(doc);                }            }        } catch (Exception e) {            e.printStackTrace();        }    }}

mongodb集群可以正常插入和删除数据；

但当slave和master节点在集群中切换时，就会出现异常

2019-10-23 20:21:05,972 WARN [org.mongodb.driver.connection] - Got socket exception on connection [connectionId{localValue:4, serverValue:15}] to 172.19.32.142:27017. All connections to 172.19.32.142:27017 will be closed.com.mongodb.MongoSocketReadException: Prematurely reached end of streamat com.mongodb.connection.SocketStream.read(SocketStream.java:88)at com.mongodb.connection.InternalStreamConnection.receiveResponseBuffers(InternalStreamConnection.java:494)at com.mongodb.connection.InternalStreamConnection.receiveMessage(InternalStreamConnection.java:224)at com.mongodb.connection.UsageTrackingInternalConnection.receiveMessage(UsageTrackingInternalConnection.java:96)at com.mongodb.connection.DefaultConnectionPool$PooledConnection.receiveMessage(DefaultConnectionPool.java:440)at com.mongodb.connection.WriteCommandProtocol.receiveMessage(WriteCommandProtocol.java:262)at com.mongodb.connection.WriteCommandProtocol.execute(WriteCommandProtocol.java:104)at com.mongodb.connection.InsertCommandProtocol.execute(InsertCommandProtocol.java:67)at com.mongodb.connection.InsertCommandProtocol.execute(InsertCommandProtocol.java:37)at com.mongodb.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:168)at com.mongodb.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:289)at com.mongodb.connection.DefaultServerConnection.insertCommand(DefaultServerConnection.java:118)at com.mongodb.operation.MixedBulkWriteOperation$Run$2.executeWriteCommandProtocol(MixedBulkWriteOperation.java:465)at com.mongodb.operation.MixedBulkWriteOperation$Run$RunExecutor.execute(MixedBulkWriteOperation.java:656)at com.mongodb.operation.MixedBulkWriteOperation$Run.execute(MixedBulkWriteOperation.java:411)at com.mongodb.operation.MixedBulkWriteOperation$1.call(MixedBulkWriteOperation.java:177)at com.mongodb.operation.MixedBulkWriteOperation$1.call(MixedBulkWriteOperation.java:168)at com.mongodb.operation.OperationHelper.withConnectionSource(OperationHelper.java:426)at com.mongodb.operation.OperationHelper.withConnection(OperationHelper.java:417)at com.mongodb.operation.MixedBulkWriteOperation.execute(MixedBulkWriteOperation.java:168)at com.mongodb.operation.MixedBulkWriteOperation.execute(MixedBulkWriteOperation.java:74)at com.mongodb.Mongo.execute(Mongo.java:845)at com.mongodb.Mongo$2.execute(Mongo.java:828)at com.mongodb.MongoCollectionImpl.executeSingleWriteRequest(MongoCollectionImpl.java:550)at com.mongodb.MongoCollectionImpl.insertOne(MongoCollectionImpl.java:317)at com.mongodb.MongoCollectionImpl.insertOne(MongoCollectionImpl.java:307)at mongotest.TestMongoDBReplSet.main(TestMongoDBReplSet.java:39)

报错:com.mongodb.MongoSocketReadException: Prematurely reached end of stream

此时使用try-catch{}捕获异常，并重连mongodb集群。(具体代码不再给出，请参考redis集群自己的经验！)

---

5.建立mongodb集群，总结常见问题

1.常见警告WARNING提示可以忽略

构建完成后，进入任何mongo容器执行mongo

root@a10c7a769a4a:/# mongo MongoDB shell version v3.4.23connecting to: mongodb://127.0.0.1:27017年MongodB server version: 3.4.23Server has startup warnings: 2019-10-23T08:25:41.590+0000 I CONTROL  [initandlisten] 2019-10-23T08:25:41.591+0000 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.2019-10-23T08:25:41.591+0000 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.2019-10-23T08:25:41.591+0000 I CONTROL  [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.2019-10-23T08:25:41.591+0000 I CONTROL  [initandlisten] 2019-10-23T08:25:41.591+0000 I CONTROL  [initandlisten] 2019-10-23T08:25:41.591+0000 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.2019-10-23T08:25:41.591+0000 I CONTROL  [initandlisten] **        We suggest setting it to 'never'2019-10-23T08:25:41.591+0000 I CONTROL  [initandlisten] 2019-10-23T08:25:41.591+0000 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.2019-10-23T08:25:41.591+0000 I CONTROL  [initandlisten] **        We suggest setting it to 'never'2019-10-23T08:25:41.591+0000 I CONTROL  [initandlisten]

基于安全警告，可以看到4个WARNING信息，对mongo的使用基本没有影响。

2.出现No host described in new configuration 1 for replica set annosys maps to this node

> use adminswitched to db admin> config={ _id:"annosys", members:[... {_id:0,host:'mongo-master:27017',priority:5},... {_id:1,host:'mongo-slave1:27017',priority:3},... {_id:3,host:'mongo-arbiter:27017',arbiterOnly:true}]... }{"_id" : "annosys","members" : [{"_id" : 0,"host" : "mongo-master:27017","priority" : 5},{"_id" : 1,"host" : "mongo-slave1:27017","priority" : 3},{"_id" : 3,"host" : "mongo-arbiter:27017","arbiterOnly" : true}]}> rs.initiate(config){"ok" : 0,"errmsg" : "No host described in new configuration 1 for replica set annosys maps to this node","code" : 93,"codeName" : "InvalidReplicaSetConfig"}

集群初始化失败，集群状态失败。此时，提示是配置项目中找不到host，副本集服务映射到节点

将mongo-master名称修改为真实ip：172.19.32.142。

config={ _id:"annosys", members:[{_id:0,host:'172.19.32.142:27017',priority:5},{_id:1,host:'172.19.32.142:27027',priority:3},{_id:2,host:'172.19.32.142:27037',arbiterOnly:true}]}

> rs.initiate(config){ "ok" : 1 }