How To Create a Sharded Cluster in MongoDB

Posted: 27th October 2016 by admin in all
Tags: , , , ,

MongoDB Sharding Topology

Sharding is implemented through three separate components. Each part performs a specific function:

  • Config Server: Each production sharding implementation must contain exactly three configuration servers. This is to ensure redundancy and high availability.

Config servers are used to store the metadata that links requested data with the shard that contains it. It organizes the data so that information can be retrieved reliably and consistently.

  • Query Routers: The query routers are the machines that your application actually connects to. These machines are responsible for communicating to the config servers to figure out where the requested data is stored. It then accesses and returns the data from the appropriate shard(s).

Each query router runs the "mongos" command.

  • Shard Servers: Shards are responsible for the actual data storage operations. In production environments, a single shard is usually composed of a replica set instead of a single machine. This is to ensure that data will still be accessible in the event that a primary shard server goes offline.

 

Implementing replicating sets is outside of the scope of this tutorial, so we will configure our shards to be single machines instead of replica sets. You can easily modify this if you would like to configure replica sets for your own configuration.

 

Initialize the Config Servers

The first thing we need to do is create a data directory, which is where the configuration server will store the metadata that associates location and content:

mkdir /mongo-metadata

Now, we simply have to start up the configuration server with the appropriate parameters. The service that provides the configuration server is called mongod. The default port number for this component is 27019.

We can start the configuration server with the following command:

(mongod –fork –logpath /opt/mongo-metadata/mongo1.log –configsvr –dbpath /opt/mongo-metadata –port 27019) – up Conf
Primary Conf Server
192.168.2.109:27019
192.168.2.109:27020
192.168.2.109:27021

Configure Query Router Instances

They query router service is called mongos. The default port number for this process is 27017 (but the port number in the configuration refers to the configuration server port number, which is 27019 by default).

The end result is that the query router service is started with a string like this:

mongos –fork –logpath /opt/mongos.log –configdb 192.168.2.109:27019,192.168.2.109:27020,192.168.2.109:27021

Script for this Procedure:

#!/bin/bash
#Check if no runing instances

if [-s /opt/MongoConf/mongoPID*]; then
killall mongod
killall mongos
fi


#Starting Conf Servers
/usr/bin/mongod --fork --logpath /opt/MongoConf/mongo1.log --configsvr --dbpath /opt/MongoConf/mongo-metadata/ --port 27019
echo -e "\033[31m Starting first Mongod Conf server"

/usr/bin/mongod --fork --logpath /opt/MongoConf/mongo1.log --configsvr --dbpath /opt/MongoConf/mongo-metadata1/ --port 27020
echo -e "\033[31m Starting second Mongod Conf server"

/usr/bin/mongod --fork --logpath /opt/MongoConf/mongo1.log --configsvr --dbpath /opt/MongoConf/mongo-metadata2/ --port 27021
echo -e "\033[31m Starting third Mongod Conf server"

echo -e "\033[31m Starting Mongos Balancer"
mongos --fork --logpath /opt/mongos.log --configdb 192.168.2.116:27019,192.168.2.116:27020,192.168.2.116:27021

echo -e "\033[31m Connect to Mongo with - mongo --host 192.168.2.116 --port 27017"

 

Add Shards to the Cluster

Shards Servers:
(mongod –fork –logpath /opt/data/instance2/monshard2.log –dbpath /opt/data/instance2 –port 27023)
(mongod –fork –logpath /opt/data/instance1/monshard1.log –dbpath /opt/data/instance1 –port 27022)

192.168.2.116:27022 ->sh.addShard( "192.168.2.116:27022" )
192.168.2.116:27023 ->sh.addShard( "192.168.2.116:27023" )

You can check your Sharding Status with – db.printShardingStatus()

--- Sharding Status --- 
  sharding version: {
    "_id" : 1,
    "version" : 3,
    "minCompatibleVersion" : 3,
    "currentVersion" : 4,
    "clusterId" : ObjectId("53317aefca1ba9ba232b949e")
}
  shards:
    {  "_id" : "shard0000",  "host" : "127.0.0.1:27000" }
    {  "_id" : "shard0001",  "host" : "127.0.0.1:27001" }
  databases:
    {  "_id" : "admin",  "partitioned" : false,  "primary" : "config" }
    {  "_id" : "test",  "partitioned" : false,  "primary" : "shard0000" }

 

Thats it. Cluster is configured and ready to work.

*