clj-kafka/README.md

# clj-kafka

Clojure library for [Kafka](http://incubator.apache.org/kafka/).

Current build status: [![Build Status](https://travis-ci.org/pingles/clj-kafka.png)](https://travis-ci.org/pingles/clj-kafka)

Development is against the 0.8 release of Kafka. The protocols for 0.7 and 0.8 are incompatible so this will only work when connecting to a 0.8 cluster. Earlier releases of clj-kafka support the earlier 0.7 release if you need it.

## Installing

Add the following to your [Leiningen](http://github.com/technomancy/leiningen) `project.clj`:

![clj-kafka version](https://clojars.org/clj-kafka/latest-version.svg)

## Usage

### Producer

Discovery of Kafka brokers from Zookeeper:

```clj
(brokers {"zookeeper.connect" "127.0.0.1:2181"})
;; ({:host "localhost", :jmx_port -1, :port 9999, :version 1})
```

```clj
(use 'clj-kafka.producer)

(def p (producer {"metadata.broker.list" "localhost:9999"
                  "serializer.class" "kafka.serializer.DefaultEncoder"
                  "partitioner.class" "kafka.producer.DefaultPartitioner"}))

(send-message p (message "test" (.getBytes "this is my message")))
```

### Zookeeper Consumer

The Zookeeper consumer uses broker information contained within Zookeeper to consume messages. This consumer also allows the client to automatically commit consumed offsets so they're not retrieved again.

```clj
(use 'clj-kafka.consumer.zk)
(use 'clj-kafka.core)

(def config {"zookeeper.connect" "localhost:2182"
             "group.id" "clj-kafka.consumer"
             "auto.offset.reset" "smallest"
             "auto.commit.enable" "false"})

(with-resource [c (consumer config)]
  shutdown
  (take 2 (messages c "test")))
```

## License

Copyright &copy; 2013 Paul Ingles

Distributed under the Eclipse Public License, the same as Clojure.

## Thanks

YourKit is kindly supporting this open source project with its full-featured Java Profiler. YourKit, LLC is the creator of innovative and intelligent tools for profiling Java and .NET applications. Take a look at YourKit's leading software products:
[YourKit Java Profiler](http://www.yourkit.com/java/profiler/index.jsp) and
[YourKit .NET Profiler](http://www.yourkit.com/.net/profiler/index.jsp).
first commit 2012-01-22 23:01:36 +00:00			`# clj-kafka`

update the README following the 0.8 changes over the weekend 2013-06-10 08:17:53 +00:00			`Clojure library for [Kafka](http://incubator.apache.org/kafka/).`
first commit 2012-01-22 23:01:36 +00:00
update README to reflect recent changes 2013-12-15 21:01:28 +00:00			`Current build status: [![Build Status](https://travis-ci.org/pingles/clj-kafka.png)](https://travis-ci.org/pingles/clj-kafka)`
Update README; pull with-resource into core; add producer 2012-01-26 10:56:32 +00:00
update README to reflect recent changes 2013-12-15 21:01:28 +00:00			`Development is against the 0.8 release of Kafka. The protocols for 0.7 and 0.8 are incompatible so this will only work when connecting to a 0.8 cluster. Earlier releases of clj-kafka support the earlier 0.7 release if you need it.`
add travis-ci build badge to README 2013-05-15 08:27:57 +00:00
Update README; pull with-resource into core; add producer 2012-01-26 10:56:32 +00:00			`## Installing`

			Add the following to your [Leiningen](http://github.com/technomancy/leiningen) `project.clj`:

add clj-kafka latest version badge to README 2014-05-14 09:22:06 +00:00			`![clj-kafka version](https://clojars.org/clj-kafka/latest-version.svg)`
Update README; pull with-resource into core; add producer 2012-01-26 10:56:32 +00:00
first commit 2012-01-22 23:01:36 +00:00			`## Usage`

Update README; pull with-resource into core; add producer 2012-01-26 10:56:32 +00:00			`### Producer`

update the README following the 0.8 changes over the weekend 2013-06-10 08:17:53 +00:00			`Discovery of Kafka brokers from Zookeeper:`
Update README; pull with-resource into core; add producer 2012-01-26 10:56:32 +00:00
			```clj
fixed typo 2013-06-12 21:40:56 +00:00			`(brokers {"zookeeper.connect" "127.0.0.1:2181"})`
update the README following the 0.8 changes over the weekend 2013-06-10 08:17:53 +00:00			`;; ({:host "localhost", :jmx_port -1, :port 9999, :version 1})`
Update README; pull with-resource into core; add producer 2012-01-26 10:56:32 +00:00			```
first commit 2012-01-22 23:01:36 +00:00
clojure syntax readme 2012-01-22 23:24:41 +00:00			```clj
update the README following the 0.8 changes over the weekend 2013-06-10 08:17:53 +00:00			`(use 'clj-kafka.producer)`
Update README; pull with-resource into core; add producer 2012-01-26 10:56:32 +00:00
update the README following the 0.8 changes over the weekend 2013-06-10 08:17:53 +00:00			`(def p (producer {"metadata.broker.list" "localhost:9999"`
			`"serializer.class" "kafka.serializer.DefaultEncoder"`
			`"partitioner.class" "kafka.producer.DefaultPartitioner"}))`
first commit 2012-01-22 23:01:36 +00:00
Fix producer example and lein version string 2013-12-17 20:25:11 +00:00			`(send-message p (message "test" (.getBytes "this is my message")))`
clojure syntax readme 2012-01-22 23:24:41 +00:00			```
first commit 2012-01-22 23:01:36 +00:00
Update README; pull with-resource into core; add producer 2012-01-26 10:56:32 +00:00			`### Zookeeper Consumer`

			`The Zookeeper consumer uses broker information contained within Zookeeper to consume messages. This consumer also allows the client to automatically commit consumed offsets so they're not retrieved again.`

			```clj
			`(use 'clj-kafka.consumer.zk)`
			`(use 'clj-kafka.core)`

update the README following the 0.8 changes over the weekend 2013-06-10 08:17:53 +00:00			`(def config {"zookeeper.connect" "localhost:2182"`
			`"group.id" "clj-kafka.consumer"`
			`"auto.offset.reset" "smallest"`
			`"auto.commit.enable" "false"})`
Update README; pull with-resource into core; add producer 2012-01-26 10:56:32 +00:00
			`(with-resource [c (consumer config)]`
			`shutdown`
Remove queue from zk consumer To ensure client code has explicit control over the reading of messages This change removes the ability to listen to multiple topics at once, as it is not feasible to build this functionality in at the library level while still giving client code total control and knowledge over which messages have been read / what the current offset is. 2014-02-24 09:29:22 +00:00			`(take 2 (messages c "test")))`
update README for a 0.0.5-0.7-SNAPSHOT release 2012-05-18 11:38:44 +00:00			```

first commit 2012-01-22 23:01:36 +00:00			`## License`

update readme and project.clj for 0.8 2013-06-06 16:09:27 +00:00			`Copyright © 2013 Paul Ingles`
first commit 2012-01-22 23:01:36 +00:00
			`Distributed under the Eclipse Public License, the same as Clojure.`
add thanks to yourkit in README 2014-04-30 09:01:42 +00:00
			`## Thanks`

			`YourKit is kindly supporting this open source project with its full-featured Java Profiler. YourKit, LLC is the creator of innovative and intelligent tools for profiling Java and .NET applications. Take a look at YourKit's leading software products:`
			`[YourKit Java Profiler](http://www.yourkit.com/java/profiler/index.jsp) and`
			`[YourKit .NET Profiler](http://www.yourkit.com/.net/profiler/index.jsp).`