Paul Ingles
79544297fe
clean the kafka log directory before using the broker in tests
2013-06-07 09:01:38 +01:00
Paul Ingles
72f8daed9e
add system-time to implement Time interface for testing
2013-06-07 08:53:01 +01:00
Paul Ingles
f661446647
start work on adding some tests; update message tests for 0.8 api
2013-06-07 08:33:51 +01:00
Paul Lam
276903b4ee
update consumer
2013-06-06 17:22:54 +01:00
Paul Lam
398f002e74
update readme and project.clj for 0.8
2013-06-06 17:09:27 +01:00
Paul Ingles
e97faeb733
releasing 0.0.7 with pipe size, thanks @DAddYE
2013-05-15 09:32:20 +01:00
Paul Ingles
1485d4845d
Merge pull request #7 from DAddYE/patch-1
...
Prevent Out of Memory with large streams
2013-05-15 01:31:44 -07:00
Paul Ingles
b7abaa16ee
add travis-ci build badge to README
2013-05-15 09:27:57 +01:00
Paul Ingles
242e57d5b3
add travis-ci yaml
2013-05-15 09:27:25 +01:00
Davide D'Agostino
2383459067
Prevent Out of Memory with large streams
...
This version caps the queue and prevent OOM and obviously uses much less memory.
My app that use some like:
(doseq [group (partition resolution (kafka/messages consumer topic))
Where resolution is a value of ~60000
Without this patch the memory usage was constantly around ~4gb
After dealing with OOMs and inspecting all vm snapshots I come that the problem was in the pipe, I checked your code then the linked article and was confirmed (in one comment) with the same solution.
2013-05-15 01:17:31 -06:00
Paul Ingles
231a739e66
release 0.0.6-0.7 with producer type hints to avoid reflection
2013-05-03 10:09:26 +01:00
Paul Ingles
7485a60472
change type hint in message
2013-05-03 10:08:47 +01:00
Paul Bergeron
787897402b
Add type annotations for producer
2013-04-08 16:21:27 -07:00
Paul Ingles
3ce92430bb
release 0.0.5-0.7
2012-12-18 10:24:26 +00:00
Paul Ingles
914a0e2389
add ToMessage protocol to help convert objects to kafka.message.Message objects
2012-10-04 09:46:45 +01:00
Paul Ingles
ab2cefe1df
lein 2 compatible project.clj
2012-10-04 09:46:32 +01:00
Paul Ingles
6dfb60fcb6
update README for a 0.0.5-0.7-SNAPSHOT release
2012-05-18 12:38:44 +01:00
Paul Ingles
1fca1666ec
use threading macro to tidy consumer.zk/messages
2012-05-18 12:28:58 +01:00
Paul Ingles
f9574228db
little refactoring in consumer.zk/messages
2012-05-18 12:28:03 +01:00
Paul Ingles
8fc0df82e3
Change handling of multiple topics- assoc a topic key that names where the message came from... not so keen on having parser/handler functions being injected
2012-05-18 12:26:07 +01:00
Paul Ingles
eb50e39b58
Revert "Update the example + allow the zookeeper consumer to specify additional functions that can be used to interpret the messages before they're aggregated into the sequence"
...
This reverts commit cdc32becff
.
2012-05-18 12:24:09 +01:00
Paul Ingles
cdc32becff
Update the example + allow the zookeeper consumer to specify additional functions that can be used to interpret the messages before they're aggregated into the sequence
2012-05-18 12:22:36 +01:00
Paul Ingles
55eefc1314
refactor consumer.zk/messages a little
2012-05-18 11:20:19 +01:00
Paul Ingles
7a3d348089
allow consumer.zk/messages to be passed multiple topics and aggregate into single seq
2012-05-18 11:02:22 +01:00
Paul Ingles
f8c72e9e75
Add topics to the consumer.zk ns, provides a list of all registered topics
2012-03-14 16:07:10 +00:00
Paul Ingles
9840461b7e
bump version- fix for payload data extraction
2012-03-14 12:11:39 +00:00
Paul Ingles
cc139ef82b
Merge branch 'master' of github.com:pingles/clj-kafka
2012-03-14 12:11:26 +00:00
Paul Ingles
df01625a56
fix a bug with extracting the byte array of the payload: didn't honour the length of the payload, just extracted the underlying array.
2012-03-14 12:10:39 +00:00
Paul Ingles
d4963d0795
Merge pull request #1 from strongh/patch-1
...
Update README.md
2012-03-10 01:13:15 -08:00
Homer Strong
9606b5c8ec
Update README.md
2012-03-09 18:15:15 -08:00
Paul Ingles
49e0cb5f7f
Add a note about Kafka binaries
2012-01-26 11:04:00 +00:00
Paul Ingles
9d6681cb48
add link to the autodoc API docs
2012-01-26 11:00:38 +00:00
Paul Ingles
0fb2673049
updated autodoc
2012-01-26 10:59:48 +00:00
Paul Ingles
d6d9d52402
as-bytes doesnt exist
2012-01-26 10:59:00 +00:00
Paul Ingles
efee73f42d
Update README; pull with-resource into core; add producer
2012-01-26 10:56:32 +00:00
Paul Ingles
9ddb351353
Add namespaces for connecting with the ZookeeperConnector and SimpleConsumer consumers; updated version to 0.0.2-0.7-SNAPSHOT
2012-01-26 10:20:44 +00:00
Paul Ingles
89032ffc4b
clojure syntax readme
2012-01-22 23:24:41 +00:00
Paul Ingles
df8cb2f747
first commit
2012-01-22 23:01:36 +00:00