Getting started with Apache Zookeeper...

It is a well known fact that building reliable and stable distributed applications requires considerable amount of effort and it's not that easy to maintain the properties such as High scalability, Availability, Transparency, performance of these systems. Developing high-quality, flexible, and interoperable software for distributed systems naturally involves challenging complexities and failures, since it is not much evolved and still so many areas to be explored. 

An interesting blog about "Why distributed systems are hard to program" can be found here.

Middle-ware is the bridge that connects application program and the communication infrastructure for basic message passing and support for reliable channels across different hardware and software platforms etc. 

"Apache ZooKeeper is an effort to develop and maintain an open-source server which enables highly reliable distributed coordination."

It is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. Although zookeeper doesn't completely prevent partial failures, it provides a set of tools to build distributed applications which can handle failures safely.

Zookeeper Deployment steps..

System Requirements

Supported platfroms

• GNU/Linux  - development and production platform for both server and client.
  
• Sun Solaris   - development and production platform for both server and client.
  
• FreeBSD    -  development and production platform for clients only.
                        (Java NIO selector support in the FreeBSD JVM is broken.)
  
• Win32        -   development platform only for both server and client.
  
• MacOSX   -   development platform only for both server and client.
  

 Required tools

• Sun Java, release 1.6 (JDK 6) or greater
• Apache Ant
• Apache Subversion(svn) or Git

Setting up the environment

(Linux is used as the development platform for rest of the guide)

• Install the Sun JDK 6
• Use a native package managing system like APT or any other popular front-end for apt to install tools listed below. (eg.Synaptic  package manager)
• otherwise you can download these packages from the web and install them manually.
  
• Use these commands in the Terminal to install ,
$ sudo add-apt-repository ppa:ferramroberto/java sudo apt-get update sudo apt-get install sun-java6-jdk sun-java6-jre  
• set the JAVA_HOME system variable
export JAVA_HOME=/usr/lib/jvm/java-6-sun-1.6.0.26 export PATH=$PATH:$JAVA_HOME/bin

• If your system has installed more than one version of Java such as open jdk and sun-jdk, configure the default java version of the system as sun-java6-jdk by entering the following command in a terminal window and selecting the correct choice.
  $ sudo update-alternatives --config java
• Verify the installation using
$ javac -version $ java -version
• Terminal out put should be like,
java version "1.6.0_26" Java(TM) SE Runtime Environment (build 1.6.0_26-b03) Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)

•  Install Apache ant

$ sudo apt-get install ant

•  Verify the installation using

$ ant -version

•  Install subversion

$ sudo apt-get install subversion

• Verify the installation using

$ svn help

•  If you intend to use git as the version control system

$ sudo apt-get install git 

         More details about setting up "Git" in Linux environment can be found here

• Set the Java heap size. 

• Since zookeeper uses a in-memory data model, swapping can seriously degrade its performance. So it is very important to set the java heap size according to the memory utilization of the system.

• To set the heap size first open the configuration fire, .bashrc located in home directory,
$ sudo vim /home/user/.bashrc

• Adding this line to the end of the file will set the heap size to 1GB(min),2GB(max)  $ export JAVA_OPTS="-Xms1g -Xmx2g or $ export JAVA_OPTS="-Xms1024m -Xmx2048m
   

Checking out the source

• Using subversion
• enter the command below in the terminal to get a working copy of zookeeper source code
  

$ svn checkout https://svn.apache.org/repos/asf/zookeeper/trunk zookeeper-trunk

• Using Git
• This will give the source code of zookeeper in read-only mode. That means you can do the changes to the source code in your local copy only. If you intend to make changes in the code and contribute developing the zookeeper you will have to create a git account and setup a  personal repository in the git-hub first. Commit(pull) request to the original repository can be made through this account. Get more details here.

$ git clone https://github.com/apache/zookeeper.git zookeeper-trunk  

Building the source

Zookeeper build process is developed using the build scripting tool called ant. Apache Ant is one of the heavily used build tool in Java development projects and very popular among open source community. Ant can automate tasks such as compiling the source code, building deployment packages and automatic dependency checking etc. Ant uses a script called "build.xml", which descibes how to carry out the compiling and building process. Detailed tutorial about "How to use ant" can be found here.

-->

Go to the top level directory of the zookeeper trunk.
Use the command below to list out all the targets defined in the build.xml.

$ ant -p

Terminal out put should be something like this.

Buildfile: /home/user/checkouts/zookeeper/build.xml Main targets: bin-package Build binary distribution binary Make tarball without source and documentation call-test-cppunit to execute cppunit tests checkstyle Run optional third-party tool targets clean Clean. Delete the build files, and their directories clean-eclipse Clean eclipse files clover Instrument the Unit tests using Clover. compile-native Make C binding deb Make deb package docs Generate forrest-based documentation. eclipse Create eclipse project files javadoc Generate javadoc javadoc-dev Generate javadoc for zookeeper developers package Build distribution package-native Make C binding tarball releaseaudit Release Audit activities rpm Make rpm package tar Make release tarball test to run core and contrib tests test-contrib to run contrib tests test-cppunit to run cppunit test cases Default target: jar

These targets can be invoked using

$ ant <target>

As a example, to build the binary distribution simply use,

$ ant bin-package

Tips and Tricks... 

If you are working behind a http proxy server, configuring things can be bit more complicated.

• checking out the source using subversion
• you need to set the proxy server settings for the subversion client in,
• $HOME/.subversion/servers  or
• /etc/subversion/servers 

                 file, under [global] settings. More details can be found here.

• Checking out the source using git 

$ git clone http://github.com/apache/zookeeper.git zookeeper-trunk

• Building source using ant, online 
• If you have set the proxy server settings in the system, you need to enforce them by using, 

$ ant -autoproxy [-args]