NBServer - Non-blocking servers in Java made easy

main | news | status | download | faq | gettingStarted | javadoc

Importance of Java as a language and runtime platform for server application grows with every day. A fundamental trait of a server application is that it services multiple clients concurrently. The only way to achieve concurrency before JDK 1.4 was to allocate a separate thread of execution for servicing every connection. While this model is quite easy to implement, it contains inherent scalability problems. First, a server that maps one connection to one thread can not serve more concurrent connections than it can allocate threads. Second, while threads - from the developer's standpoint - provide a convenient virtualization of available CPU resources, they are costly both in terms of space (each thread requires a separate call stack) and time (context switching a CPU between threads consumes time). All these factors impose limits on both the number of connections the server can process at any given time, as well as on the effective throughput. Last, but not least the threads will potentially spend a significant part of their processing time blocked in I/O operations. This makes the server vulnerable to a particular kind of denial-of-service attacks, where a malicious client can bog down the server by opening many connections to it (therefore allocating all available threads), and then send it a completely valid request extremely slowly (say, one byte per minute). This effect can be caused even with otherwise benevolent clients sitting behind narrow bandwidth connections.

If you are concerned with any of these problems (support for extremely large number of connections, throughput maximization, and protection from service degradation due to slow clients), you should write your servers in a non-blocking fashion. This means a radical paradigm shift - instead of allocating a dedicated thread to serve a connection, a single thread (or eventually several threads) service a large number of connections using an event-driven mechanism. In the event-driven architecture, one thread watches multiple network sockets, and when one or more sockets are ready to be read from or written to, the thread recieves an event and gets the chance to service those connections that became ready. However, this architecture assumes the availability of a non-blocking I/O library, since a crucial requirement is that the thread must never block on an I/O operation.

Also, the way network protocol is implemented in a non-blocking world is drastically different than in the blocking world. With the blocking, one-thread-per connection paradigm, you encapsulate a network protocol in a procedural way, that is you write a code that is executed on a thread that is dedicated to executing the code for its single connection. You can store much of the processing state on the stack: local variables, call arguments, and the code execution path itself. On the contrary, in the non-blocking world, your code is invoked to process one or two packets of data at a time and then it returns control. Therefore, you can't contain any state on the stack, as you would  Essentially, you must write a finite-state machine where events (incoming packet and empty outgoing network buffer) drive state transitions.

Furthermore, the sole availability of non-blocking I/O is only one of requirements for a non-blocking server. You still have to attend to many subtle details of content buffering, event handling loops, various bookkeeping operations, etc.. 

 The NBServer framework will hide the most tedious aspects of writing a non-blocking network service from you, namely those named above as "subtle details" - it has facilities for content buffering, management of event-handling loops (with thread pooling), bookkeeping operations (such as registering a connection with read and write event loops after it is accepted in the accept event loop, etc.).

All you have to write is an implementation of an event handler interface that defines the tryToWrite() and tryToRead() methods. Also, the tryToWrite() has a stock implementation suitable for majority of protocols that streams buffered content to the client one packet at a time (see content buffering below).

The content buffering issue might not seem obvious, but consider the following: your network protocol implementation will send content (that is, a response or responses) to the client. You don't want to deal with issues of transferring the response one packet at a time; ideally you just want an output stream that you simply write your response to and which never blocks - if you write more data to it than can get transmitted through the socket immediately, the stream buffers it and transfers it to the socket when it becomes ready for writing. This is just what the provided NonblockingPipe class does in conjunction with the stock implementation of tryToWrite() event handler.

The bad news is that non-blocking I/O support is introduced in Java with the version 1.4, which is currently in a release candidate (RC) state. The very bad news is that the JDK 1.4 RC has many bugs in the NIO library, and many of these bugs are so serious that they downright prevent the correct operation of an implementation that is otherwise coced to the JDK specs. In our opinion, this even more underlines the importance of a framework like NBServer, since we can (hopefully) hide all the required workarounds in the framework code, and adjust these workarounds as the JDK evolves toward final release transparently for protocol implementations.

Our commitment, however is to have an elegant framework that is not messed up with hacky workarounds for bugs. In this spirit, the current codebase is written to the specs inasmuch as possible, and exhibits correct behavior with only minimum of hacks and compromises. If you are a member of Sun's Java Developer Connection (and if you are not, you can register for free), you can cast your vote for these bugs, potentially advancing their resolution (bugs with more votes get higher priority inside Sun) . Most serious bugs are 4531726 and 4615008, which will cause the server to cease reading data under heavy load and prevent it from detecting closed connections altough 4503092 (which limits multiplexing to 63 connections per thread on Windows platforms)  is also notable. There are also other bugs that were since fixed, but the fixes will not appear until the next beta release whose schedule is not known. The bugs in the previous beta were so severe, it even prevented the NBServer to pass its own JUnit tests! Fortunately, this is no longer the case.

The NBServer framework is available under an Apache-style license. This roughly means that you can use and modify it in either binary or source form for commercial and non-commercial purposes as long as you give proper credit to the copyright holder. You can view the actual text of the license here. You can either download releases,or you can access the latest bits from the CVS repository.

If you would like to get involved with this project, please visit the project page at SourceForge. You will find the CVS repository, mailing lists and other collaboration resources there.

There is an open source third-party non-blocking I/O library (NBIO) and an associated service framework (SEDA) written on top of it by Matt Welsh, an expert that also works on the non-blocking I/O in JDK 1.4. NBIO is primarily developed and tested on Linux and Solaris, and is in a beta labeled "unstable" on Windows platforms (that said, it can't be much worse than the current status of JDK 1.4 NIO package...) SEDA (short for Staged Event Driven Architecture) is a generic framework for decoupling processing and threads, and one of its subsystems named Sandstorm is a foundation for building event-driven network servers. SEDA also features many advanced facilities such as per-stage resource throttling. in comparison, NBServer pursues a different philisophy: it's not a generic event-driven framework, instead it's a framework specialized for writing non-blocking network services on top of JDK 1.4 non-blocking I/O classes.