Difference between revisions of "Apache Kafka"
Jump to navigation
Jump to search
(56 intermediate revisions by 3 users not shown) | |||
Line 1: | Line 1: | ||
− | [[wikipedia:Apache Kafka]] ([[2011]] | + | [[wikipedia:Apache Kafka]] (Jan [[2011]], [https://trends.google.com/trends/explore?date=today%205-y&q=%2Fm%2F0zmynvd Trends], [[Scala]], [[Java]]) [[Stream processing]] |
− | + | * https://kafka.apache.org/ | |
+ | * Installation: | ||
+ | ** macOS: <code>[[brew install kafka]] && [[brew install zookeeper]] && [[brew install kcat]]</code> | ||
+ | ** Docker: <code>[[docker pull]] apache/kafka:3.7.0 && [[docker run -d --name kafka]] -p [[9092]]:9092 apache/kafka:3.7.0 && [[docker exec]] -it kafka bash</code> | ||
+ | ** Kubernetes: [[Helm install kafka]] | ||
+ | * Status: <code>[[brew services info kafka]] && [[brew services info zookeeper]]</code> | ||
+ | * Configuration: <code>[[/etc/kafka/]]</code> | ||
+ | ** <code>[[server.properties]]</code> | ||
+ | * Cloud: [[Amazon Managed Streaming for Apache Kafka (MSK)]] (2018) | ||
+ | * Components: | ||
+ | ** [[Kafka controller]] | ||
− | + | == Binaries == | |
+ | [[kafka-acls]] | ||
+ | [[kafka-broker-api-versions]] | ||
+ | [[kafka-cluster]] | ||
+ | [[kafka-configs]] | ||
+ | [[kafka-console-consumer]] | ||
+ | [[kafka-console-producer]] | ||
+ | kafka-consumer-groups | ||
+ | kafka-consumer-perf-test | ||
+ | kafka-delegation-tokens | ||
+ | kafka-delete-records | ||
+ | kafka-dump-log | ||
+ | [[kafka-features]] | ||
+ | kafka-leader-election | ||
+ | [[kafka-log-dirs]] | ||
+ | kafka-metadata-shell | ||
+ | kafka-mirror-maker | ||
+ | kafka-preferred-replica-election | ||
+ | kafka-producer-perf-test | ||
+ | [[kafka-reassign-partitions]] | ||
+ | kafka-replica-verification | ||
+ | kafka-run-class | ||
+ | [[kafka-server-start]] | ||
+ | kafka-server-stop | ||
+ | [[kafka-storage]] | ||
+ | kafka-streams-application-reset | ||
+ | [[kafka-topics]] | ||
+ | kafka-verifiable-consumer | ||
+ | kafka-verifiable-producer | ||
− | == | + | == News == |
− | + | * Oct 2022, v3.3.0 https://www.confluent.io/blog/apache-kafka-3-3-0-new-features-and-updates/ | |
− | + | ** [[KRaft]] mode available for new cluster production ready | |
− | |||
− | |||
− | |||
− | |||
− | kafka- | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
+ | == Activities == | ||
+ | * Create a [[Topic]] using <code>[[kafka-topics --create]]</code> and send <code>([[kafka-console-producer]])</code> and receive messages <code>([[kafka-console-consumer]])</code> | ||
== Related terms == | == Related terms == | ||
* [[PubSub]] | * [[PubSub]] | ||
− | * [[Apache Zookeeper]] | + | * [[Apache Zookeeper]] distributed and hierarchical [[key-value]] store. |
+ | * [[AWS Kinesis]] | ||
+ | * [[Sentry]] | ||
+ | * [[Conduktor]] platform | ||
+ | * [[Zipkin]] | ||
+ | * [[Scala]] | ||
+ | * [[apt-get install kafkacat]] | ||
+ | * [[KRaft]] ([[Kafka Raft]]) | ||
+ | * [[Cloudera]] | ||
+ | * [[Linkedin Cruise-control]] | ||
+ | * [[Kafka Streams]] | ||
+ | * [[KIP-405: Kafka Tiered Storage]] | ||
== See also == | == See also == | ||
+ | * {{kafka-topics}} | ||
+ | * {{AWS MSK}} | ||
* {{Kafka}} | * {{Kafka}} | ||
− | |||
− | |||
[[Category:Apache]] | [[Category:Apache]] | ||
+ | [[Category:Kafka]] | ||
+ | [[Category:MQ]] |
Latest revision as of 14:44, 4 April 2024
wikipedia:Apache Kafka (Jan 2011, Trends, Scala, Java) Stream processing
- https://kafka.apache.org/
- Installation:
- macOS:
brew install kafka && brew install zookeeper && brew install kcat
- Docker:
docker pull apache/kafka:3.7.0 && docker run -d --name kafka -p 9092:9092 apache/kafka:3.7.0 && docker exec -it kafka bash
- Kubernetes: Helm install kafka
- macOS:
- Status:
brew services info kafka && brew services info zookeeper
- Configuration:
/etc/kafka/
- Cloud: Amazon Managed Streaming for Apache Kafka (MSK) (2018)
- Components:
Binaries[edit]
kafka-acls kafka-broker-api-versions kafka-cluster kafka-configs kafka-console-consumer kafka-console-producer kafka-consumer-groups kafka-consumer-perf-test kafka-delegation-tokens kafka-delete-records kafka-dump-log kafka-features kafka-leader-election kafka-log-dirs kafka-metadata-shell kafka-mirror-maker kafka-preferred-replica-election kafka-producer-perf-test kafka-reassign-partitions kafka-replica-verification kafka-run-class kafka-server-start kafka-server-stop kafka-storage kafka-streams-application-reset kafka-topics kafka-verifiable-consumer kafka-verifiable-producer
News[edit]
- Oct 2022, v3.3.0 https://www.confluent.io/blog/apache-kafka-3-3-0-new-features-and-updates/
- KRaft mode available for new cluster production ready
Activities[edit]
- Create a Topic using
kafka-topics --create
and send(kafka-console-producer)
and receive messages(kafka-console-consumer)
Related terms[edit]
- PubSub
- Apache Zookeeper distributed and hierarchical key-value store.
- AWS Kinesis
- Sentry
- Conduktor platform
- Zipkin
- Scala
- apt-get install kafkacat
- KRaft (Kafka Raft)
- Cloudera
- Linkedin Cruise-control
- Kafka Streams
- KIP-405: Kafka Tiered Storage
See also[edit]
kafka-topics [ --list ]
- Kafka, AWS MSK,
aws kafka
- Kafka,
kafka-cluster -h
,kafka-configs, kafka-features
,kafka-topics, kafka-consumer-groups, kafka-storage, kafka-server-start, kafka-server-stop, kafka-reassign-partitions, kafka-topics, kafka-console-producer
, Amazon MSK, Kafka Connect, 9092,server.properties
, KRaft, Kafka changelog
Advertising: