Sunday, January 15, 2012

New pub/sub system-Apache Kafka

Apache Kafka, it is from linkedin with a lot of specific characters. It aims at providing a unified stream for both real-time and offline consumption. It stores data in the file sequentially without random access. The status of consumer is stored in client side and the consumer works in pull way. It uses zero-copy method like sendfile (FileChannel.transferTo/transferFrom) to reduce copy cost. It really looks a good design. I think it's worth having a try. Here is a video about this. http://sna-projects.com/blog/2011/08/kafka/

No comments:

Post a Comment