分类目录

Howto: ZeroMQ (ØMQ) & PyZMQ Install Guide

Homepage: http://www.zeromq.org

ØMQ \zeromq\:
Ø  The socket library that acts as a concurrency framework.
Ø  Carries messages across inproc, IPC, TCP, and multicast.
Ø  Connect N-to-N via fanout, pubsub, pipeline, request-reply.
Ø  Fast enough for clustered products and supercomputing.
Ø  Asynch I/O for scalable multicore message-passing apps.
Ø  Large and active open source community.
Ø  20+ languages including C, C++, Java, .NET, Python.
Ø  Most OSes including Linux, Windows, OS X.
Ø  LGPL free software with full commercial support from iMatix.

1. ZeroMQ (ØMQ) Installation

# wget http://download.zeromq.org/zeromq-2.1.6.tar.gz
# tar zxvf zeromq-2.1.6.tar.gz
# cd zeromq-2.1.6
# ./configure
# make
# make install
# vi /etc/ld.so.conf.d/zmq.conf
/usr/local/lib
# ldconfig /etc/ld.so.conf

2. PyZMQ Installation

# easy_install-2.6 pyzmq

3. Study ZMQ

消息队列的适用场合
* 可靠传输
* 分布式计算
* 多进程通信

4. 同类产品

ZeroMQ 关注
RabbitMQ(Erlang) 关注
Apache QPID
Red Hat Enterprise MRG
Fedora AMQP Infrastructure
OpenAMQ

Refer to:
1. ZeroMQ Guide
http://zguide.zeromq.org/page:all
2. AMQP和ZeroMQ的分析
https://docs.google.com/present/view?id=df85d3vm_316dbhtg3f7&pli=1

Howto: Transifex Installation and Configuration

Translation service: http://www.transifex.net/
Open source software: http://www.transifex.org/

1. Installation (CentOS 5.2 & Python 2.6 & Django 1.2.4)

# easy_install -U transifex
Note 1: some related rpm packages will be installed by yum
Note 2: the following packages will be installed by python
./Jinja2-2.5.5-py2.6.egg
./Pygments-1.4-py2.6.egg
./PIL-1.1.7-py2.6-linux-i686.egg
./polib-0.6.3-py2.6.egg
./httplib2-0.6.0-py2.6.egg
./userprofile-0.7_r422_correct_validation-py2.6.egg
./Markdown-2.0.3-py2.6.egg
./pycurl-7.19.0-py2.6-linux-i686.egg
./ctypes-1.0.2-py2.6-linux-i686.egg
./transifex-1.0.0-py2.6.egg

2. Configuration for development

# cd /usr/lib/python2.6/site-packages/transifex-1.0.0-py2.6.egg/transifex
Note: running the following command by python
./manage.py txcreatedirs # Create necessary directories
./manage.py syncdb # Setup DB tables, create superuser
./manage.py migrate # Setup more DB tables
./manage.py txlanguages # Create a standard set of languages
./manage.py txcreatenoticetypes # Create a standard set of notice types
./manage.py build_static # Copy all the addons files to the static dir

3. Running for development

# cd /usr/lib/python2.6/site-packages/transifex-1.0.0-py2.6.egg/transifex
Note: running the following command by python
./manage.py runserver 0.0.0.0:8000 # Start the development server

4. Configuration for apache

# vi /etc/httpd/conf.d/transifex.conf
< IfModule !python_module>
LoadModule python_module modules/python26-mod_python.so
< /IfModule>
< Directory "/var/www/manual/mod/python26-mod_python">
< Files *.html>
SetHandler default-handler
< /Files>
< /Directory>
< Location "/transifex/">
SetHandler python-program
PythonHandler django.core.handlers.modpython
SetEnv DJANGO_SETTINGS_MODULE settings
PythonOption django.root /transifex
PythonDebug On
PythonPath "['/usr/lib/python2.6/site-packages/transifex-1.0.0-py2.6.egg/transifex'] + sys.path"
< /Location>

Alias /site_media /usr/lib/python2.6/site-packages/transifex-1.0.0-py2.6.egg/transifex/site_media
< Directory /usr/lib/python2.6/site-packages/transifex-1.0.0-py2.6.egg/transifex/site_media>
Order Deny,Allow
Deny from all
# Allow from 127.0.0.1
Allow from 192.168.1.0/24
< /Directory>

Note: change some file permission to apache, e.g. the database file and its parent directory.

Refer to:
Installation Guide

Howto: MongoDB Installation Guide

MongoDB (from “humongous”) is a scalable, high-performance, open source, document-oriented database, written in C++.

Homepage: http://www.mongodb.org/

1. Download the package

# uname -a
Linux hzserver1 2.6.18-8.el5 #1 SMP Fri Jan 26 14:15:21 EST 2007 i686 i686 i386 GNU/Linux
# mkdir -p /home/mongo
# cd /home/mongo
# wget http://fastdl.mongodb.org/linux/mongodb-linux-i686-1.8.1.tgz
# tar zxvf mongodb-linux-i686-1.8.1.tgz
# tree mongodb-linux-i686-1.8.1
mongodb-linux-i686-1.8.1
|-- GNU-AGPL-3.0
|-- README
|-- THIRD-PARTY-NOTICES
`-- bin
|-- bsondump
|-- mongo
|-- mongod
|-- mongodump
|-- mongoexport
|-- mongofiles
|-- mongoimport
|-- mongorestore
|-- mongos
|-- mongosniff
`-- mongostat

1 directory, 14 files

Note: 32-bit builds are limited to around 2GB of data. Recommend 64-bit system.

2. Install the package
Official Architectural Overview
Current servers:
server1(192.168.1.67):
shard1: /home/mongo/data/shard11
shard2: /home/mongo/data/shard21
config1: /home/mongo/data/config
mongos
server2(192.168.1.68):
shard1: /home/mongo/data/shard12
shard2: /home/mongo/data/shard22
config2: /home/mongo/data/config
mongos
Note: server1 & server2 set up to Replica Sets.

1) server1 tree:

# tree /home/mongo
mongo
|-- bin
| |-- bsondump
| |-- mongo
| |-- mongo.start.sh
| |-- mongod
| |-- mongodump
| |-- mongoexport
| |-- mongofiles
| |-- mongoimport
| |-- mongorestore
| |-- mongos
| |-- mongosniff
| `-- mongostat
`-- data
|-- config
| |-- _tmp
| |-- config.0
| |-- config.1
| |-- config.log
| |-- config.ns
| |-- diaglog.4da6a3ac
| |-- diaglog.4da6a4e1
| |-- diaglog.4da6ac00
| `-- mongod.lock
|-- mongos.log
|-- shard11
| |-- local.0
| |-- local.1
| |-- local.2
| |-- local.ns
| |-- mongod.lock
| `-- shard11.log
`-- shard21
|-- local.0
|-- local.1
|-- local.2
|-- local.ns
|-- mongod.lock
`-- shard21.log

6 directories, 33 files

2) server2 tree:

[root@hzserver2 home]# tree mongo
mongo
|-- bin
| |-- bsondump
| |-- mongo
| |-- mongo.start.sh
| |-- mongod
| |-- mongodump
| |-- mongoexport
| |-- mongofiles
| |-- mongoimport
| |-- mongorestore
| |-- mongos
| |-- mongosniff
| `-- mongostat
`-- data
|-- config
| |-- config.log
| |-- diaglog.4da6a371
| |-- diaglog.4da6a4f4
| |-- diaglog.4da6a573
| |-- diaglog.4da6acee
| `-- mongod.lock
|-- mongos.log
|-- shard12
| |-- local.0
| |-- local.1
| |-- local.2
| |-- local.ns
| |-- mongod.lock
| `-- shard12.log
`-- shard22
|-- local.0
|-- local.1
|-- local.2
|-- local.ns
|-- mongod.lock
`-- shard22.log

5 directories, 31 files

3. Configuring the Replica Sets
1) Start shard1 replica set

# /home/mongo/bin/mongo -port 10000 -host 192.168.1.67
> config = {_id:'shard1',members:[{_id: 0, host: '192.168.1.67:10000'},{_id: 1, host: '192.168.1.68:10000'}]}
> rs.initiate(config);

2) Start shard2 replica set

# /home/mongo/bin/mongo -port 10000 -host 192.168.1.67
> config = {_id:'shard2',members:[{_id: 0, host: '192.168.1.67:10001'},{_id: 1, host: '192.168.1.68:10001'}]}
> rs.initiate(config);

4. Mongo start scripts
1) server1

# cat /home/mongo/bin/mongo.start.sh
/home/mongo/bin/mongod -shardsvr -replSet shard1 -port 10000 -dbpath /home/mongo/data/shard11 -oplogSize 100 -logpath /home/mongo/data/shard11/shard11.log -fork
/home/mongo/bin/mongod -shardsvr -replSet shard2 -port 10001 -dbpath /home/mongo/data/shard21 -oplogSize 100 -logpath /home/mongo/data/shard21/shard21.log -fork
/home/mongo/bin/mongod -configsvr -dbpath /home/mongo/data/config -port 20000 -logpath /home/mongo/data/config/config.log -logappend -fork
/home/mongo/bin/mongos -configdb 192.168.1.67:20000 -port 30000 -chunkSize 5 -logpath /home/mongo/data/mongos.log -logappend -fork

2) server2

# cat /home/mongo/bin/mongo.start.sh
/home/mongo/bin/mongod -shardsvr -replSet shard1 -port 10000 -dbpath /home/mongo/data/shard12 -oplogSize 100 -logpath /home/mongo/data/shard12/shard12.log -fork
/home/mongo/bin/mongod -shardsvr -replSet shard2 -port 10001 -dbpath /home/mongo/data/shard22 -oplogSize 100 -logpath /home/mongo/data/shard22/shard22.log -fork
/home/mongo/bin/mongod -configsvr -dbpath /home/mongo/data/config -port 20000 -logpath /home/mongo/data/config/config.log -logappend -fork
/home/mongo/bin/mongos -configdb 192.168.1.67:20000 -port 30000 -chunkSize 5 -logpath /home/mongo/data/mongos.log -logappend -fork

Note: mongos only use 1 or 3 configdb.

5. Configuring the Shard Cluster

# /home/mongo/bin/mongo 192.168.1.67:30000/admin
> db
admin
> db.runCommand({addshard:"shard1/192.168.1.67:10000,192.168.1.68:10000", name:"s1", maxsize:20480});
{ "shardAdded" : "s1", "ok" : 1 }
> db.runCommand({addshard:"shard2/192.168.1.67:10001,192.168.1.68:10001", name:"s2", maxsize:20480});
{ "shardAdded" : "s2", "ok" : 1 }
> db.runCommand({listshards:1})
{
"shards" : [
{
"_id" : "s1",
"host" : "shard1/192.168.1.67:10000,192.168.1.68:10000"
},
{
"_id" : "s2",
"host" : "shard2/192.168.1.67:10001,192.168.1.68:10001"
}
],
"ok" : 1
}
> db.runCommand({enablesharding:"test_database"})
{ "ok" : 1 }

Refer to:
1. 配置mongodb分片群集(sharding cluster)
2. MongoDB shard replica config/ 分片 复制 配置