About Me

My photo
Senior Java Engineer, Wavecell Pte Ltd. Singapore

Tuesday, January 17, 2012

Getting started with Apache Zookeeper...

It is a well known fact that building reliable and stable distributed applications requires considerable amount of effort and it's not that easy to maintain the properties such as High scalability, Availability, Transparency, performance of these systems. Developing high-quality, flexible, and interoperable software for distributed systems naturally involves challenging complexities and failures, since it is not much evolved and still so many areas to be explored. 

An interesting blog about "Why distributed systems are hard to program" can be found here.

Middle-ware is the bridge that connects application program and the communication infrastructure for basic message passing and support for reliable channels across different hardware and software platforms etc. 

"Apache ZooKeeper is an effort to develop and maintain an open-source server which enables highly reliable distributed coordination."

It is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services. Although zookeeper doesn't completely prevent partial failures, it provides a set of tools to build distributed applications which can handle failures safely.

Zookeeper Deployment steps..

System Requirements

Supported platfroms
  • GNU/Linux  - development and production platform for both server and client.
  • Sun Solaris   - development and production platform for both server and client.
  • FreeBSD    -  development and production platform for clients only.
                          (Java NIO selector support in the FreeBSD JVM is broken.)
  • Win32        -   development platform only for both server and client.
  • MacOSX   -   development platform only for both server and client.

 Required tools

Setting up the environment
(Linux is used as the development platform for rest of the guide)
  • Install the Sun JDK 6
    • Use a native package managing system like APT or any other popular front-end for apt to install tools listed below. (eg.Synaptic  package manager)
    • otherwise you can download these packages from the web and install them manually.
    • Use these commands in the Terminal to install ,
    • set the JAVA_HOME system variable
    • If your system has installed more than one version of Java such as open jdk and sun-jdk, configure the default java version of the system as sun-java6-jdk by entering the following command in a terminal window and selecting the correct choice.
    • Verify the installation using
    • Terminal out put should be like,
  •  Install Apache ant
    •  Verify the installation using

  •  Install subversion
    • Verify the installation using

  •  If you intend to use git as the version control system
     
         More details about setting up "Git" in Linux environment can be found here
  • Set the Java heap size. 
    • Since zookeeper uses a in-memory data model, swapping can seriously degrade its performance. So it is very important to set the java heap size according to the memory utilization of the system.
      • To set the heap size first open the configuration fire, .bashrc located in home directory,
      • Adding this line to the end of the file will set the heap size to 1GB(min),2GB(max) 
       
Checking out the source
  • Using subversion
    • enter the command below in the terminal to get a working copy of zookeeper source code

    • Using Git
      • This will give the source code of zookeeper in read-only mode. That means you can do the changes to the source code in your local copy only. If you intend to make changes in the code and contribute developing the zookeeper you will have to create a git account and setup a  personal repository in the git-hub first. Commit(pull) request to the original repository can be made through this account. Get more details here.
 

Building the source
Zookeeper build process is developed using the build scripting tool called ant. Apache Ant is one of the heavily used build tool in Java development projects and very popular among open source community. Ant can automate tasks such as compiling the source code, building deployment packages and automatic dependency checking etc. Ant uses a script called "build.xml", which descibes how to carry out the compiling and building process. Detailed tutorial about "How to use ant" can be found here.

-->
Go to the top level directory of the zookeeper trunk.
Use the command below to list out all the targets defined in the build.xml.

Terminal out put should be something like this.

These targets can be invoked using

As a example, to build the binary distribution simply use,

Tips and Tricks... 

If you are working behind a http proxy server, configuring things can be bit more complicated.
    • checking out the source using subversion
      • you need to set the proxy server settings for the subversion client in,
        • $HOME/.subversion/servers  or
        • /etc/subversion/servers 
                 file, under [global] settings. More details can be found here.
    • Checking out the source using git 
    • Building source using ant, online 
      • If you have set the proxy server settings in the system, you need to enforce them by using, 


10 comments:


Handling POST request using akka-http in a akka-actor system

Akka-htp is a well known module for providing full server side as well as client side http stack currently supporting Scala and Java langua...