{"id":991,"date":"2018-10-28T12:40:14","date_gmt":"2018-10-28T04:40:14","guid":{"rendered":"http:\/\/www.rain1024.com\/?p=991"},"modified":"2023-08-07T20:57:25","modified_gmt":"2023-08-07T12:57:25","slug":"article136","status":"publish","type":"post","link":"http:\/\/rain1024.com\/index.php\/2018\/10\/28\/article136\/","title":{"rendered":"Java\u5b9e\u73b0Kafka\u751f\u4ea7\u8005\u6d88\u8d39\u8005\u529f\u80fd"},"content":{"rendered":"<h1>Java\u5b9e\u73b0Kafka\u751f\u4ea7\u8005\u6d88\u8d39\u8005\u529f\u80fd<\/h1>\n<h2>\u597d\u4e45\u6ca1\u6709\u66f4\u65b0\u535a\u5ba2\uff0c\u6700\u8fd1\u5b66\u7684\u4e1c\u897f\u5f88\u591a\uff0c\u4f46\u4e00\u76f4\u5fd9\u7684\u6ca1\u6709\u65f6\u95f4\u53bb\u5199\uff0c\u5148\u8865\u5145\u4e00\u7bc7kafka\u7684\uff0c\u6700\u57fa\u672c\u7684\u529f\u80fd\u4f7f\u7528\uff0c\u4e0d\u5f97\u4e0d\u611f\u53f9\u5927\u6570\u636e\u786e\u5b9e\u96be\uff0c\u5373\u4f7f\u53ea\u8bf4\u4e00\u4e2a\u7b80\u5355\u7684\u529f\u80fd\uff0c\u4e4b\u524d\u4e5f\u9700\u8981\u94fa\u57ab\u5f88\u591a\u5b8c\u6210\u7684\u529f\u80fd\uff0c\u6bd4\u5982\u8fd9\u7bc7\u535a\u5ba2\u7684\u524d\u63d0\u662f\uff0c\u4f60\u5df2\u7ecf\u5b89\u88c5\u4e86\u865a\u62df\u673a\uff0c\u91cc\u9762\u914d\u7f6e\u4e86Hadoop\u751f\u6001\u7ec4\u4ef6zookeeper\uff0c\u5b89\u88c5\u914d\u7f6e\u4e86kafka\uff0c\u5b66\u4f1a\u4f7f\u7528Maven\uff0cspringboot\u7b49\u4e9b\u6280\u672f\uff0c\u800c\u4e0d\u662f\u76f4\u63a5\u62ff\u6765\u4ee3\u7801\u5c31\u53ef\u4ee5\u590d\u5236\u7c98\u8d34\u3002<\/h2>\n<h2>\u4fdd\u8bc1\u4f60\u7684\u865a\u62df\u673a\u662f\u53ef\u4ee5ping\u901a\u7684\uff0chmaster\u662f\u6211\u5728host\u4e2d\u914d\u7f6e\u7684\u865a\u62df\u673aIP\uff0c\u53ef\u4ee5\u4fee\u6539\u4e3a\u81ea\u5df1\u7684\u3002<\/h2>\n<h2>\u5728shell\u4e2d\u5f00\u542f\u4e24\u4e2a\u7a97\u53e3\uff0c\u6d4b\u8bd5\u7ec8\u7aef\u4e2d\u7684producer\u548cconsumer\u80fd\u5426\u5728\u540c\u4e00\u4e2atopic\u4e2d\u4f20\u9012\u6d88\u606f\u3002<\/h2>\n<h2>\u542f\u52a8kafka<\/h2>\n<ul>\n<li><code>bin\/kafka-server-start.sh \/config\/server.properties<\/code><\/li>\n<\/ul>\n<h2>\u4f7f\u7528Kafka(\u5355\u8282\u70b9\u5355broker) \u2022 \u521b\u5efatopic: zk<\/h2>\n<p>\u2022 kafka-topics.sh --create --zookeeper hadoop000:2181 --replication-factor 1 --partitions 1 --topic hello_topic<\/p>\n<h2>\u67e5\u770b\u6240\u6709topic<\/h2>\n<p>\u2022 kafka-topics.sh --list --zookeeper hadoop000:2181<\/p>\n<h2>\u53d1\u9001\u6d88\u606f: broker<\/h2>\n<p>\u2022 kafka-console-producer.sh --broker-list hadoop000:9092 --topic hello_topic<\/p>\n<h2>\u6d88\u8d39\u6d88\u606f: zk<\/h2>\n<p>\u2022 kafka-console-consumer.sh --zookeeper hadoop000:2181 --topic hello_topic --from-beginning<\/p>\n<p><img decoding=\"async\" src=\"http:\/\/cos.rain1024.com\/markdown\/kafka1.png\" alt=\"\" \/><\/p>\n<h2><img decoding=\"async\" src=\"http:\/\/cos.rain1024.com\/markdown\/kafka2.png\" alt=\"\" \/><\/h2>\n<h2>Maven\u4f9d\u8d56<\/h2>\n<pre><code>&lt;dependency&gt;\n      &lt;groupId&gt;org.apache.kafka&lt;\/groupId&gt;\n      &lt;artifactId&gt;kafka-clients&lt;\/artifactId&gt;\n      &lt;version&gt;0.9.0.0&lt;\/version&gt;\n    &lt;\/dependency&gt;\n\n    &lt;dependency&gt;\n      &lt;groupId&gt;org.apache.kafka&lt;\/groupId&gt;\n      &lt;artifactId&gt;kafka_2.11&lt;\/artifactId&gt;\n      &lt;version&gt;0.9.0.0&lt;\/version&gt;\n    &lt;\/dependency&gt;\n<\/code><\/pre>\n<h2>ProducerDemo<\/h2>\n<pre><code>import java.util.Arrays;\nimport java.util.Properties;\n\nimport org.apache.kafka.clients.consumer.ConsumerRecord;\nimport org.apache.kafka.clients.consumer.ConsumerRecords;\nimport org.apache.kafka.clients.consumer.KafkaConsumer;\n\npublic class ConsumerDemo {\n    public static void main(String[] args){\n        Properties properties = new Properties();\n        properties.put(\"bootstrap.servers\", \"hmaster:9092\");\n        properties.put(\"group.id\", \"group-1\");\n        properties.put(\"enable.auto.commit\", \"true\");\n        properties.put(\"auto.commit.interval.ms\", \"1000\");\n        properties.put(\"auto.offset.reset\", \"earliest\");\n        properties.put(\"session.timeout.ms\", \"30000\");\n        properties.put(\"key.deserializer\", \"org.apache.kafka.common.serialization.StringDeserializer\");\n        properties.put(\"value.deserializer\", \"org.apache.kafka.common.serialization.StringDeserializer\");\n\n        KafkaConsumer&lt;String, String&gt; kafkaConsumer = new KafkaConsumer&lt;&gt;(properties);\n        kafkaConsumer.subscribe(Arrays.asList(\"test\"));\n        while (true) {\n            ConsumerRecords&lt;String, String&gt; records = kafkaConsumer.poll(100);\n            for (ConsumerRecord&lt;String, String&gt; record : records) {\n                System.out.printf(\"offset = %d, value = %s\", record.offset(), record.value());\n                System.out.println();\n            }\n        }\n\n    }\n}\n<\/code><\/pre>\n<h2>Consumer<\/h2>\n<pre><code>import java.util.Arrays;\nimport java.util.Properties;\n\nimport org.apache.kafka.clients.consumer.ConsumerRecord;\nimport org.apache.kafka.clients.consumer.ConsumerRecords;\nimport org.apache.kafka.clients.consumer.KafkaConsumer;\n\npublic class ConsumerDemo {\n    public static void main(String[] args){\n        Properties properties = new Properties();\n        properties.put(\"bootstrap.servers\", \"hmaster:9092\");\n        properties.put(\"group.id\", \"group-1\");\n        properties.put(\"enable.auto.commit\", \"true\");\n        properties.put(\"auto.commit.interval.ms\", \"1000\");\n        properties.put(\"auto.offset.reset\", \"earliest\");\n        properties.put(\"session.timeout.ms\", \"30000\");\n        properties.put(\"key.deserializer\", \"org.apache.kafka.common.serialization.StringDeserializer\");\n        properties.put(\"value.deserializer\", \"org.apache.kafka.common.serialization.StringDeserializer\");\n\n        KafkaConsumer&lt;String, String&gt; kafkaConsumer = new KafkaConsumer&lt;&gt;(properties);\n        kafkaConsumer.subscribe(Arrays.asList(\"test\"));\n        while (true) {\n            ConsumerRecords&lt;String, String&gt; records = kafkaConsumer.poll(100);\n            for (ConsumerRecord&lt;String, String&gt; record : records) {\n                System.out.printf(\"offset = %d, value = %s\", record.offset(), record.value());\n                System.out.println();\n            }\n        }\n\n    }\n}\n\n<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"<p>Java\u5b9e\u73b0Kafka\u751f\u4ea7\u8005\u6d88\u8d39\u8005\u529f\u80fd \u597d\u4e45\u6ca1\u6709\u66f4\u65b0\u535a\u5ba2\uff0c\u6700\u8fd1\u5b66\u7684\u4e1c\u897f\u5f88\u591a\uff0c\u4f46\u4e00\u76f4\u5fd9\u7684\u6ca1\u6709\u65f6\u95f4\u53bb\u5199\uff0c\u5148\u8865\u5145\u4e00\u7bc7\u2026 <span class=\"read-more\"><a href=\"http:\/\/rain1024.com\/index.php\/2018\/10\/28\/article136\/\">Read More &raquo;<\/a><\/span><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[52],"class_list":["post-991","post","type-post","status-publish","format-standard","hentry","category-kafka","tag-kafka"],"_links":{"self":[{"href":"http:\/\/rain1024.com\/index.php\/wp-json\/wp\/v2\/posts\/991","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/rain1024.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/rain1024.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/rain1024.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/rain1024.com\/index.php\/wp-json\/wp\/v2\/comments?post=991"}],"version-history":[{"count":1,"href":"http:\/\/rain1024.com\/index.php\/wp-json\/wp\/v2\/posts\/991\/revisions"}],"predecessor-version":[{"id":1396,"href":"http:\/\/rain1024.com\/index.php\/wp-json\/wp\/v2\/posts\/991\/revisions\/1396"}],"wp:attachment":[{"href":"http:\/\/rain1024.com\/index.php\/wp-json\/wp\/v2\/media?parent=991"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/rain1024.com\/index.php\/wp-json\/wp\/v2\/categories?post=991"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/rain1024.com\/index.php\/wp-json\/wp\/v2\/tags?post=991"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}