2012-01-22 23:01:36 +00:00
# clj-kafka
2013-06-10 08:17:53 +00:00
Clojure library for [Kafka ](http://incubator.apache.org/kafka/ ).
2012-01-22 23:01:36 +00:00
2013-12-15 21:01:28 +00:00
Current build status: [![Build Status ](https://travis-ci.org/pingles/clj-kafka.png )](https://travis-ci.org/pingles/clj-kafka)
2012-01-26 10:56:32 +00:00
2013-12-15 21:01:28 +00:00
Development is against the 0.8 release of Kafka. The protocols for 0.7 and 0.8 are incompatible so this will only work when connecting to a 0.8 cluster. Earlier releases of clj-kafka support the earlier 0.7 release if you need it.
2013-05-15 08:27:57 +00:00
2012-01-26 10:56:32 +00:00
## Installing
Add the following to your [Leiningen ](http://github.com/technomancy/leiningen ) `project.clj` :
```clj
2013-12-17 20:25:11 +00:00
[clj-kafka "0.1.2-0.8"]
2012-01-26 10:56:32 +00:00
```
2012-01-22 23:01:36 +00:00
## Usage
2012-01-26 10:56:32 +00:00
### Producer
2013-06-10 08:17:53 +00:00
Discovery of Kafka brokers from Zookeeper:
2012-01-26 10:56:32 +00:00
```clj
2013-06-12 21:40:56 +00:00
(brokers {"zookeeper.connect" "127.0.0.1:2181"})
2013-06-10 08:17:53 +00:00
;; ({:host "localhost", :jmx_port -1, :port 9999, :version 1})
2012-01-26 10:56:32 +00:00
```
2012-01-22 23:01:36 +00:00
2012-01-22 23:24:41 +00:00
```clj
2013-06-10 08:17:53 +00:00
(use 'clj-kafka.producer)
2012-01-26 10:56:32 +00:00
2013-06-10 08:17:53 +00:00
(def p (producer {"metadata.broker.list" "localhost:9999"
"serializer.class" "kafka.serializer.DefaultEncoder"
"partitioner.class" "kafka.producer.DefaultPartitioner"}))
2012-01-22 23:01:36 +00:00
2013-12-17 20:25:11 +00:00
(send-message p (message "test" (.getBytes "this is my message")))
2012-01-22 23:24:41 +00:00
```
2012-01-22 23:01:36 +00:00
2012-01-26 10:56:32 +00:00
### Zookeeper Consumer
The Zookeeper consumer uses broker information contained within Zookeeper to consume messages. This consumer also allows the client to automatically commit consumed offsets so they're not retrieved again.
```clj
(use 'clj-kafka.consumer.zk)
(use 'clj-kafka.core)
2013-06-10 08:17:53 +00:00
(def config {"zookeeper.connect" "localhost:2182"
"group.id" "clj-kafka.consumer"
"auto.offset.reset" "smallest"
"auto.commit.enable" "false"})
2012-01-26 10:56:32 +00:00
(with-resource [c (consumer config)]
shutdown
2014-02-24 09:29:22 +00:00
(take 2 (messages c "test")))
2012-05-18 11:38:44 +00:00
```
2012-01-22 23:01:36 +00:00
## License
2013-06-06 16:09:27 +00:00
Copyright © 2013 Paul Ingles
2012-01-22 23:01:36 +00:00
Distributed under the Eclipse Public License, the same as Clojure.