yogsototh/clj-kafka
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
112 commits 1 branch 4 tags 157 KiB
Commit graph

47 commits

Author SHA1 Message Date
Paul Ingles
8acfb69df6 add broker-list to help build metadata.broker.list strings; change clj-kafka.zk/brokers to return an empty list if zookeeper is available but no kafka brokers are registered. 2014-04-15 13:54:29 +01:00
prepor
4ff8334299 Do not use iterator-seq with KafkaStream 2014-03-19 22:31:49 +09:00
Russell
c606237b1c Remove queue from zk consumer 
To ensure client code has explicit control over the reading of messages
This change removes the ability to listen to multiple topics at once,
as it is not feasible to build this functionality in at the library
level while still giving client code total control and knowledge over
which messages have been read / what the current offset is.
 2014-02-24 12:36:03 +00:00
Paul Ingles
98a039115b update tests and fix controller fn to extract the controller brokerid 2013-12-15 20:55:05 +00:00
Paul Ingles
422ccf2b84 update project to use Kafka's 0.8-beta1 release 2013-09-19 11:15:38 +01:00
Paul Ingles
52c61f12c1 await kafka shutdown to minimise errors reported in logs 2013-06-10 20:13:53 +01:00
Paul Ingles
f997dcfcb0 rename :zookeeper-connect in to-clojure to :connect - it is the details to connect directly to the broker 2013-06-10 15:07:33 +01:00
Paul Lam
e71246eda1 add latest-topic-offset to simple consumer 2013-06-10 13:54:09 +01:00
Paul Lam
6a547f5616 get topics from zk 2013-06-10 12:13:03 +01:00
Paul Lam
27ba361517 zk/controller to get leader node 2013-06-10 11:56:49 +01:00
Paul Lam
c46c916ca0 zk ns to hold zookeeper methods 2013-06-10 11:36:37 +01:00
Paul Lam
602be6f061 add :broker-id for to-clojure on Broker 2013-06-10 11:23:18 +01:00
Paul Lam
7978821026 topic-meta-data pass in topic and returns brokers with that topic 2013-06-10 11:20:00 +01:00
Paul Lam
dd1cdeeb86 mv test ns consumer.clj into consumer/zk.clj 2013-06-10 10:47:18 +01:00
Paul Lam
2f24376609 use zookeeper-port in tests 2013-06-10 09:23:07 +01:00
Paul Ingles
b9d3c7a069 update travis-ci to run only expectations 2013-06-09 09:18:19 +01:00
Paul Ingles
d9ab7ba672 update travis-ci to run both tests and expectations; started rewriting tests as expectations 2013-06-09 08:39:46 +01:00
Paul Ingles
1d92367158 deleting empty test ns 2013-06-09 08:24:15 +01:00
Paul Ingles
5ba3d9b67b delete MessageValue protocol- clients should know whether they're providing a byte array or not and they can implement their own protocols. 2013-06-09 08:17:48 +01:00
Paul Ingles
24f182c824 change producer send-message to receive a KeyedMessage rather than topic/value; rename keyed-message to just message and have pulled message-value out of send-message: clients should know whether they're providing a byte array or not. 2013-06-09 08:15:03 +01:00
Paul Ingles
f75c4b339a change log4j to append to ./logs/kafka.log; tidy the zk consumer test 2013-06-09 07:51:34 +01:00
Paul Ingles
257c25293d remove println; move test broker config into a Var 2013-06-09 07:43:24 +01:00
Paul Ingles
76ede151a4 pull paren up 2013-06-08 11:52:07 +01:00
Paul Ingles
1e014986c5 refactor test.utils/with-test-broker to receive config map specifying ports + topic name 2013-06-08 11:49:52 +01:00
Paul Ingles
be2cd0329f add queue-capacity option for zookeeper messages: can be used to limit LinkedBlockingQueue queue size with large messages 2013-06-07 22:11:42 +01:00
Paul Ingles
84c813ae48 add message piping back; to-clojure conversion to KafkaMessage added by default 2013-06-07 22:03:11 +01:00
Paul Ingles
57a9f17dfa rename with-broker to with-test-broker 2013-06-07 21:55:11 +01:00
Paul Ingles
6437dcd18d delete unused to-clojure definition 2013-06-07 21:09:02 +01:00
Paul Ingles
8643566f5c add log4j properties to reduce logging in dev/test; delete tmp directory before test run too 2013-06-07 20:58:23 +01:00
Paul Lam
c01ffa4a96 added brokers in producer to retrieve list of brokers from zookeeper, 
cool
 2013-06-07 16:35:48 +01:00
Paul Ingles
989965a2d1 add example for simple consumer 2013-06-07 14:57:41 +01:00
Paul Ingles
c44ddf8dba rename test 2013-06-07 14:19:41 +01:00
Paul Ingles
4c10bf6c48 both tests are the same 2013-06-07 14:18:46 +01:00
Paul Ingles
9f23aec1df fix cleanup after test run; add MessagePayload protocol and second test 2013-06-07 14:17:38 +01:00
Paul Ingles
fa0de3cae0 update code to pass test; consumes messages but clj-kafka.consumer.zk/messages can't map the contents for us... 2013-06-07 13:55:17 +01:00
Paul Ingles
f4a1ebe88a more work on updating for 0.8 api; clj-kafka.test.consumer is WIP and hangs currently when creating the messages sequence 2013-06-07 13:03:55 +01:00
Paul Ingles
2492aee358 Revert "work on updating the api to work against 0.8" 
This reverts commit 5286bbfbb6.
 2013-06-07 10:48:08 +01:00
Paul Ingles
5286bbfbb6 work on updating the api to work against 0.8 2013-06-07 10:23:13 +01:00
Paul Ingles
f3d01cbfba remove println 2013-06-07 09:43:14 +01:00
Paul Ingles
72cec16cd6 move with-broker testing stuff into test.utils ns 2013-06-07 09:42:04 +01:00
Paul Ingles
a89a454d3b create embedded zookeeper server 2013-06-07 09:22:55 +01:00
Paul Ingles
79544297fe clean the kafka log directory before using the broker in tests 2013-06-07 09:01:38 +01:00
Paul Ingles
72f8daed9e add system-time to implement Time interface for testing 2013-06-07 08:53:01 +01:00
Paul Ingles
f661446647 start work on adding some tests; update message tests for 0.8 api 2013-06-07 08:33:51 +01:00
Paul Ingles
914a0e2389 add ToMessage protocol to help convert objects to kafka.message.Message objects 2012-10-04 09:46:45 +01:00
Paul Ingles
df01625a56 fix a bug with extracting the byte array of the payload: didn't honour the length of the payload, just extracted the underlying array. 2012-03-14 12:10:39 +00:00
Paul Ingles
df8cb2f747 first commit 2012-01-22 23:01:36 +00:00