Difference between revisions of "Apache Kafka"
Jump to navigation
Jump to search
| (11 intermediate revisions by the same user not shown) | |||
| Line 4: | Line 4: | ||
** macOS: <code>[[brew install kafka]] && [[brew install zookeeper]] && [[brew install kcat]]</code> | ** macOS: <code>[[brew install kafka]] && [[brew install zookeeper]] && [[brew install kcat]]</code> | ||
** Docker: <code>[[docker pull]] apache/kafka:3.7.0 && [[docker run -d --name kafka]] -p [[9092]]:9092 apache/kafka:3.7.0 && [[docker exec]] -it kafka bash</code> | ** Docker: <code>[[docker pull]] apache/kafka:3.7.0 && [[docker run -d --name kafka]] -p [[9092]]:9092 apache/kafka:3.7.0 && [[docker exec]] -it kafka bash</code> | ||
| + | ** Kubernetes: [[Helm install kafka]] | ||
* Status: <code>[[brew services info kafka]] && [[brew services info zookeeper]]</code> | * Status: <code>[[brew services info kafka]] && [[brew services info zookeeper]]</code> | ||
| − | * Configuration: <code>[[server.properties]]</code> | + | * Configuration: <code>[[/etc/kafka/]]</code> |
| + | ** <code>[[server.properties]]</code> | ||
| + | * Cloud: [[Amazon Managed Streaming for Apache Kafka (MSK)]] (2018) | ||
| + | * Components: | ||
| + | ** [[Kafka controller]] | ||
== Binaries == | == Binaries == | ||
| Line 37: | Line 42: | ||
kafka-verifiable-producer | kafka-verifiable-producer | ||
| + | == News == | ||
| + | * Oct 2022, v3.3.0 https://www.confluent.io/blog/apache-kafka-3-3-0-new-features-and-updates/ | ||
| + | ** [[KRaft]] mode available for new cluster production ready | ||
== Activities == | == Activities == | ||
| Line 43: | Line 51: | ||
== Related terms == | == Related terms == | ||
* [[PubSub]] | * [[PubSub]] | ||
| − | * [[Apache Zookeeper]] | + | * [[Apache Zookeeper]] distributed and hierarchical [[key-value]] store. |
* [[AWS Kinesis]] | * [[AWS Kinesis]] | ||
* [[Sentry]] | * [[Sentry]] | ||
| Line 50: | Line 58: | ||
* [[Scala]] | * [[Scala]] | ||
* [[apt-get install kafkacat]] | * [[apt-get install kafkacat]] | ||
| − | * [[KRaft]] | + | * [[KRaft]] ([[Kafka Raft]]) |
* [[Cloudera]] | * [[Cloudera]] | ||
| + | * [[Linkedin Cruise-control]] | ||
| + | * [[Kafka Streams]] | ||
| + | * [[KIP-405: Kafka Tiered Storage]] | ||
== See also == | == See also == | ||
| − | * {{kafka topics}} | + | * {{kafka-topics}} |
* {{AWS MSK}} | * {{AWS MSK}} | ||
* {{Kafka}} | * {{Kafka}} | ||
Latest revision as of 14:44, 4 April 2024
wikipedia:Apache Kafka (Jan 2011, Trends, Scala, Java) Stream processing
- https://kafka.apache.org/
- Installation:
- macOS:
brew install kafka && brew install zookeeper && brew install kcat - Docker:
docker pull apache/kafka:3.7.0 && docker run -d --name kafka -p 9092:9092 apache/kafka:3.7.0 && docker exec -it kafka bash - Kubernetes: Helm install kafka
- macOS:
- Status:
brew services info kafka && brew services info zookeeper - Configuration:
/etc/kafka/ - Cloud: Amazon Managed Streaming for Apache Kafka (MSK) (2018)
- Components:
Binaries[edit]
kafka-acls kafka-broker-api-versions kafka-cluster kafka-configs kafka-console-consumer kafka-console-producer kafka-consumer-groups kafka-consumer-perf-test kafka-delegation-tokens kafka-delete-records kafka-dump-log kafka-features kafka-leader-election kafka-log-dirs kafka-metadata-shell kafka-mirror-maker kafka-preferred-replica-election kafka-producer-perf-test kafka-reassign-partitions kafka-replica-verification kafka-run-class kafka-server-start kafka-server-stop kafka-storage kafka-streams-application-reset kafka-topics kafka-verifiable-consumer kafka-verifiable-producer
News[edit]
- Oct 2022, v3.3.0 https://www.confluent.io/blog/apache-kafka-3-3-0-new-features-and-updates/
- KRaft mode available for new cluster production ready
Activities[edit]
- Create a Topic using
kafka-topics --createand send(kafka-console-producer)and receive messages(kafka-console-consumer)
Related terms[edit]
- PubSub
- Apache Zookeeper distributed and hierarchical key-value store.
- AWS Kinesis
- Sentry
- Conduktor platform
- Zipkin
- Scala
- apt-get install kafkacat
- KRaft (Kafka Raft)
- Cloudera
- Linkedin Cruise-control
- Kafka Streams
- KIP-405: Kafka Tiered Storage
See also[edit]
kafka-topics [ --list ]- Kafka, AWS MSK,
aws kafka - Kafka,
kafka-cluster -h,kafka-configs, kafka-features,kafka-topics, kafka-consumer-groups, kafka-storage, kafka-server-start, kafka-server-stop, kafka-reassign-partitions, kafka-topics, kafka-console-producer, Amazon MSK, Kafka Connect, 9092,server.properties, KRaft, Kafka changelog
Advertising:
