Difference between revisions of "Apache Kafka"
Jump to navigation
Jump to search
| (55 intermediate revisions by 3 users not shown) | |||
| Line 1: | Line 1: | ||
| − | [[wikipedia:Apache Kafka]] ([[2011]] | + | [[wikipedia:Apache Kafka]] (Jan [[2011]], [https://trends.google.com/trends/explore?date=today%205-y&q=%2Fm%2F0zmynvd Trends], [[Scala]], [[Java]]) [[Stream processing]] |
| − | + | * https://kafka.apache.org/ | |
| − | + | * Installation: | |
| − | * <code>[[brew install]] kafka</code> | + | ** macOS: <code>[[brew install kafka]] && [[brew install zookeeper]] && [[brew install kcat]]</code> |
| + | ** Docker: <code>[[docker pull]] apache/kafka:3.7.0 && [[docker run -d --name kafka]] -p [[9092]]:9092 apache/kafka:3.7.0 && [[docker exec]] -it kafka bash</code> | ||
| + | ** Kubernetes: [[Helm install kafka]] | ||
| + | * Status: <code>[[brew services info kafka]] && [[brew services info zookeeper]]</code> | ||
| + | * Configuration: <code>[[/etc/kafka/]]</code> | ||
| + | ** <code>[[server.properties]]</code> | ||
| + | * Cloud: [[Amazon Managed Streaming for Apache Kafka (MSK)]] (2018) | ||
| + | * Components: | ||
| + | ** [[Kafka controller]] | ||
== Binaries == | == Binaries == | ||
| − | |||
[[kafka-acls]] | [[kafka-acls]] | ||
[[kafka-broker-api-versions]] | [[kafka-broker-api-versions]] | ||
[[kafka-cluster]] | [[kafka-cluster]] | ||
[[kafka-configs]] | [[kafka-configs]] | ||
| − | + | [[kafka-console-consumer]] | |
| − | kafka-console-consumer | + | [[kafka-console-producer]] |
| − | kafka-console-producer | + | kafka-consumer-groups |
| − | kafka-consumer-groups | + | kafka-consumer-perf-test |
| − | kafka-consumer-perf-test | + | kafka-delegation-tokens |
| − | kafka-delegation-tokens | + | kafka-delete-records |
| − | kafka-delete-records | + | kafka-dump-log |
| − | kafka-dump-log | + | [[kafka-features]] |
| − | kafka-features | + | kafka-leader-election |
| − | kafka-leader-election | + | [[kafka-log-dirs]] |
| − | kafka-log-dirs | + | kafka-metadata-shell |
| − | kafka-metadata-shell | + | kafka-mirror-maker |
| − | kafka-mirror-maker | + | kafka-preferred-replica-election |
| − | kafka-preferred-replica-election | + | kafka-producer-perf-test |
| − | kafka-producer-perf-test | + | [[kafka-reassign-partitions]] |
| − | kafka-reassign-partitions | + | kafka-replica-verification |
| − | kafka-replica-verification | + | kafka-run-class |
| − | kafka-run-class | + | [[kafka-server-start]] |
| − | kafka-server-start | + | kafka-server-stop |
| − | kafka-server-stop | + | [[kafka-storage]] |
| − | kafka-storage | + | kafka-streams-application-reset |
| − | kafka-streams-application-reset | + | [[kafka-topics]] |
| − | kafka-topics | + | kafka-verifiable-consumer |
| − | kafka-verifiable-consumer | + | kafka-verifiable-producer |
| − | kafka-verifiable-producer | + | |
| − | </ | + | == News == |
| + | * Oct 2022, v3.3.0 https://www.confluent.io/blog/apache-kafka-3-3-0-new-features-and-updates/ | ||
| + | ** [[KRaft]] mode available for new cluster production ready | ||
| + | |||
| + | == Activities == | ||
| + | * Create a [[Topic]] using <code>[[kafka-topics --create]]</code> and send <code>([[kafka-console-producer]])</code> and receive messages <code>([[kafka-console-consumer]])</code> | ||
== Related terms == | == Related terms == | ||
* [[PubSub]] | * [[PubSub]] | ||
| − | * [[Apache Zookeeper]] | + | * [[Apache Zookeeper]] distributed and hierarchical [[key-value]] store. |
| + | * [[AWS Kinesis]] | ||
| + | * [[Sentry]] | ||
| + | * [[Conduktor]] platform | ||
| + | * [[Zipkin]] | ||
| + | * [[Scala]] | ||
| + | * [[apt-get install kafkacat]] | ||
| + | * [[KRaft]] ([[Kafka Raft]]) | ||
| + | * [[Cloudera]] | ||
| + | * [[Linkedin Cruise-control]] | ||
| + | * [[Kafka Streams]] | ||
| + | * [[KIP-405: Kafka Tiered Storage]] | ||
== See also == | == See also == | ||
| + | * {{kafka-topics}} | ||
| + | * {{AWS MSK}} | ||
* {{Kafka}} | * {{Kafka}} | ||
| − | |||
| − | |||
[[Category:Apache]] | [[Category:Apache]] | ||
| + | [[Category:Kafka]] | ||
| + | [[Category:MQ]] | ||
Latest revision as of 14:44, 4 April 2024
wikipedia:Apache Kafka (Jan 2011, Trends, Scala, Java) Stream processing
- https://kafka.apache.org/
- Installation:
- macOS:
brew install kafka && brew install zookeeper && brew install kcat - Docker:
docker pull apache/kafka:3.7.0 && docker run -d --name kafka -p 9092:9092 apache/kafka:3.7.0 && docker exec -it kafka bash - Kubernetes: Helm install kafka
- macOS:
- Status:
brew services info kafka && brew services info zookeeper - Configuration:
/etc/kafka/ - Cloud: Amazon Managed Streaming for Apache Kafka (MSK) (2018)
- Components:
Binaries[edit]
kafka-acls kafka-broker-api-versions kafka-cluster kafka-configs kafka-console-consumer kafka-console-producer kafka-consumer-groups kafka-consumer-perf-test kafka-delegation-tokens kafka-delete-records kafka-dump-log kafka-features kafka-leader-election kafka-log-dirs kafka-metadata-shell kafka-mirror-maker kafka-preferred-replica-election kafka-producer-perf-test kafka-reassign-partitions kafka-replica-verification kafka-run-class kafka-server-start kafka-server-stop kafka-storage kafka-streams-application-reset kafka-topics kafka-verifiable-consumer kafka-verifiable-producer
News[edit]
- Oct 2022, v3.3.0 https://www.confluent.io/blog/apache-kafka-3-3-0-new-features-and-updates/
- KRaft mode available for new cluster production ready
Activities[edit]
- Create a Topic using
kafka-topics --createand send(kafka-console-producer)and receive messages(kafka-console-consumer)
Related terms[edit]
- PubSub
- Apache Zookeeper distributed and hierarchical key-value store.
- AWS Kinesis
- Sentry
- Conduktor platform
- Zipkin
- Scala
- apt-get install kafkacat
- KRaft (Kafka Raft)
- Cloudera
- Linkedin Cruise-control
- Kafka Streams
- KIP-405: Kafka Tiered Storage
See also[edit]
kafka-topics [ --list ]- Kafka, AWS MSK,
aws kafka - Kafka,
kafka-cluster -h,kafka-configs, kafka-features,kafka-topics, kafka-consumer-groups, kafka-storage, kafka-server-start, kafka-server-stop, kafka-reassign-partitions, kafka-topics, kafka-console-producer, Amazon MSK, Kafka Connect, 9092,server.properties, KRaft, Kafka changelog
Advertising:
