Apache RocketMQ

Apache RocketMQ 是一个分布式消息和流平台，具有低延迟、高性能和可靠、万亿级容量和灵活的可扩展性。

它提供了多种功能：

消息传递模式包括发布/订阅、请求/回复和流式传输
金融级交易信息
基于DLedger Controller 的内置容错和高可用性配置选项
内置消息追踪能力，同时支持opentracing
多功能大数据和流媒体生态系统集成
按时间或偏移量追溯消息
可靠的 FIFO 和严格有序的消息传递在同一队列中
高效的拉动和推动消费模式
单队列百万级消息堆积能力
多种消息传递协议，如 gRPC、MQTT、JMS 和 OpenMessaging
灵活的分布式横向扩展部署架构
闪电般快速的批量消息交换系统
各种消息过滤机制，例如 SQL 和标签
用于隔离测试和云隔离集群的 Docker 镜像
功能丰富的管理仪表板，用于配置、指标和监控
身份验证和授权
免费的开源连接器，适用于源和接收器
轻量级实时计算

视频教程：https://www.bilibili.com/video/BV1L4411y7mn?p=1

特性

	特征	说明
	低延迟	高压下在1毫秒内响应延迟超过99.6％。
	财务导向	具有跟踪和审核功能的高可用性。
	产业可持续发展	万亿级消息容量得到保证。
	供应商中性	自最新的4.1版本以来，新的开放式分布式消息传递和流标准。
	大数据友好	具有通用集成功能的批处理传输可实现吞吐吞吐量。
	大量积累	给定足够的磁盘空间，可以在不损失性能的情况下累积邮件。

Apache RocketMQ 5.0 版本完成基本消息收发，包括 NameServer、Broker、Proxy 组件。在 5.0 版本中 Proxy 和 Broker 根据实际诉求可以分为 Local 模式和 Cluster 模式，一般情况下如果没有特殊需求，或者遵循从早期版本平滑升级的思路，可以选用Local模式。

在 Local 模式下，Broker 和 Proxy 是同进程部署，只是在原有 Broker 的配置基础上新增 Proxy 的简易配置就可以运行。
在 Cluster 模式下，Broker 和 Proxy 分别部署，即在原有的集群基础上，额外再部署 Proxy 即可。

快速入门
本段将指导您以不同方式安装 RocketMQ 的步骤。对于本地开发和测试，每个组件只会创建一个实例。

本地运行 RocketMQ
RocketMQ 可在所有主流操作系统上运行，并且只需要安装 Java JDK 8 或更高版本。要检查，请运行java -version：

$ java -version
java version “1.8.0_121”
对于 Windows 用户，点击此处下载 5.2.0 RocketMQ 二进制版本，解压到本地磁盘，例如D:\rocketmq。对于 macOS 和 Linux 用户，执行以下命令：

Download release from the Apache mirror

$ wget https://dist.apache.org/repos/dist/release/rocketmq/5.2.0/rocketmq-all-5.2.0-bin-release.zip

Unpack the release

$ unzip rocketmq-all-5.2.0-bin-release.zip
准备一个终端并切换到解压的bin目录：

$ cd rocketmq-all-5.2.0-bin-release/bin
1）启动NameServer

NameServer 将会监听0.0.0.0:9876，请确保该端口未被本地机器上的其他人使用，然后执行以下操作。

对于 macOS 和 Linux 用户：

start Name Server

$ nohup sh mqnamesrv &

check whether Name Server is successfully started

$ tail -f ~/logs/rocketmqlogs/namesrv.log
The Name Server boot success…
对于 Windows 用户，您需要先设置环境变量：

在桌面上，右键单击“计算机”图标。
从上下文菜单中选择“属性”。
单击高级系统设置链接。
单击环境变量。
添加环境ROCKETMQ_HOME=”D:\rocketmq”。
然后将目录更改为 rocketmq，输入并运行：

$ mqnamesrv.cmd
The Name Server boot success…
2）启动 Broker

对于 macOS 和 Linux 用户：

start Broker

$ nohup sh bin/mqbroker -n localhost:9876 &

check whether Broker is successfully started, eg: Broker’s IP is 192.168.1.2, Broker’s name is broker-a

$ tail -f ~/logs/rocketmqlogs/broker.log
The broker[broker-a, 192.169.1.2:10911] boot success…
对于 Windows 用户：

$ mqbroker.cmd -n localhost:9876
The broker[broker-a, 192.169.1.2:10911] boot success…
在 Docker 中运行 RocketMQ
您可以在自己的机器上的 Docker 容器内运行 RocketMQ， host网络将用于公开容器中的监听端口。

1）启动NameServer

$ docker run -it –net=host apache/rocketmq ./mqnamesrv
2）启动 Broker

$ docker run -it –net=host –mount source=/tmp/store,target=/home/rocketmq/store apache/rocketmq ./mqbroker -n localhost:9876
在 Kubernetes 中运行 RocketMQ
您还可以使用RocketMQ Operator在 Kubernetes 集群中运行 RocketMQ 集群。在操作之前，请确保kubectl您的机器上安装了相关的 kubeconfig 文件。

1）安装 CRD

install CRDs

$ git clone https://github.com/apache/rocketmq-operator
$ cd rocketmq-operator && make deploy

check whether CRDs is successfully installed

$ kubectl get crd | grep rocketmq.apache.org
brokers.rocketmq.apache.org 2022-05-12T09:23:18Z
consoles.rocketmq.apache.org 2022-05-12T09:23:19Z
nameservices.rocketmq.apache.org 2022-05-12T09:23:18Z
topictransfers.rocketmq.apache.org 2022-05-12T09:23:19Z

check whether operator is running

$ kubectl get pods | grep rocketmq-operator
rocketmq-operator-6f65c77c49-8hwmj 1/1 Running 0 93s
2）创建集群实例

create RocketMQ cluster resource

$ cd example && kubectl create -f rocketmq_v1alpha1_rocketmq_cluster.yaml

check whether cluster resources is running

$ kubectl get sts
NAME READY AGE
broker-0-master 1/1 107m
broker-0-replica-1 1/1 107m
name-service 1/1 107m
Apache RocketMQ 社区
RocketMQ Streams：基于 Apache RocketMQ 的轻量级流计算引擎。
RocketMQ Flink：Apache Flink 的 Apache RocketMQ 连接器，支持数据流和表中的源和接收器连接器。
RocketMQ API：RocketMQ protobuf 协议。
RocketMQ 客户端：基于 gRPC/protobuf 的 RocketMQ 客户端。
基于 RocketMQ Remoting 的客户端
RocketMQ 客户端 CPP
RocketMQ 客户端
RocketMQ 客户端 Python
RocketMQ 客户端 Nodejs
RocketMQ Spring：一个帮助开发人员快速将 Apache RocketMQ 与 Spring Boot 集成的项目。
RocketMQ Exporter：Prometheus 的 Apache RocketMQ 导出器。
RocketMQ Operator：提供在 Kubernetes 上运行 Apache RocketMQ 集群的方法。
RocketMQ Docker：Apache RocketMQ 的 Docker 镜像的 Git 存储库。
RocketMQ Dashboard：Apache RocketMQ 的运维控制台。
RocketMQ Connect：一种在 Apache RocketMQ 和其他系统之间可扩展且可靠地流式传输数据的工具。
RocketMQ MQTT：一种新的 MQTT 协议架构模型，基于此 Apache RocketMQ 可以更好的支持物联网设备、手机APP 等终端的消息。
RocketMQ EventBridge：EventBridge 使构建事件驱动的应用程序变得更加容易。
RocketMQ 孵化社区项目：Apache RocketMQ 的孵化社区项目，包括logappender、rocketmq-ansible、rocketmq-beats-integration、rocketmq-cloudevents-binding等。
RocketMQ 站点：Apache RocketMQ 网站的存储库。
RocketMQ E2E：用于测试 Apache RocketMQ 的项目，包括端到端、性能、兼容性测试。

Quick Start

This paragraph guides you through steps of installing RocketMQ in different ways.
For local development and testing, only one instance will be created for each component.

Run RocketMQ locally

RocketMQ runs on all major operating systems and requires only a Java JDK version 8 or higher to be installed.
To check, run java -version:

$ java -version
java version "1.8.0_121"

For Windows users, click here to download the 5.2.0 RocketMQ binary release,
unpack it to your local disk, such as D:\rocketmq.
For macOS and Linux users, execute following commands:

# Download release from the Apache mirror
$ wget https://dist.apache.org/repos/dist/release/rocketmq/5.2.0/rocketmq-all-5.2.0-bin-release.zip

# Unpack the release
$ unzip rocketmq-all-5.2.0-bin-release.zip

Prepare a terminal and change to the extracted bin directory:

$ cd rocketmq-all-5.2.0-bin-release/bin

1) Start NameServer

NameServer will be listening at 0.0.0.0:9876, make sure that the port is not used by others on the local machine, and then do as follows.

For macOS and Linux users:

### start Name Server
$ nohup sh mqnamesrv &

### check whether Name Server is successfully started
$ tail -f ~/logs/rocketmqlogs/namesrv.log
The Name Server boot success...

For Windows users, you need set environment variables first:

From the desktop, right click the Computer icon.
Choose Properties from the context menu.
Click the Advanced system settings link.
Click Environment Variables.
Add Environment ROCKETMQ_HOME="D:\rocketmq".

Then change directory to rocketmq, type and run:

$ mqnamesrv.cmd
The Name Server boot success...

2) Start Broker

For macOS and Linux users:

### start Broker
$ nohup sh bin/mqbroker -n localhost:9876 &

### check whether Broker is successfully started, eg: Broker's IP is 192.168.1.2, Broker's name is broker-a
$ tail -f ~/logs/rocketmqlogs/broker.log
The broker[broker-a, 192.169.1.2:10911] boot success...

For Windows users:

$ mqbroker.cmd -n localhost:9876
The broker[broker-a, 192.169.1.2:10911] boot success...

Run RocketMQ in Docker

You can run RocketMQ on your own machine within Docker containers,
host network will be used to expose listening port in the container.

1) Start NameServer

$ docker run -it --net=host apache/rocketmq ./mqnamesrv

2) Start Broker

$ docker run -it --net=host --mount source=/tmp/store,target=/home/rocketmq/store apache/rocketmq ./mqbroker -n localhost:9876

Run RocketMQ in Kubernetes

You can also run a RocketMQ cluster within a Kubernetes cluster using RocketMQ Operator.
Before your operations, make sure that kubectl and related kubeconfig file installed on your machine.

1) Install CRDs

### install CRDs
$ git clone https://github.com/apache/rocketmq-operator
$ cd rocketmq-operator && make deploy

### check whether CRDs is successfully installed
$ kubectl get crd | grep rocketmq.apache.org
brokers.rocketmq.apache.org                 2022-05-12T09:23:18Z
consoles.rocketmq.apache.org                2022-05-12T09:23:19Z
nameservices.rocketmq.apache.org            2022-05-12T09:23:18Z
topictransfers.rocketmq.apache.org          2022-05-12T09:23:19Z

### check whether operator is running
$ kubectl get pods | grep rocketmq-operator
rocketmq-operator-6f65c77c49-8hwmj   1/1     Running   0          93s

2) Create Cluster Instance

### create RocketMQ cluster resource
$ cd example && kubectl create -f rocketmq_v1alpha1_rocketmq_cluster.yaml

### check whether cluster resources is running
$ kubectl get sts
NAME                 READY   AGE
broker-0-master      1/1     107m
broker-0-replica-1   1/1     107m
name-service         1/1     107m

Apache RocketMQ Community

RocketMQ Streams: A lightweight stream computing engine based on Apache RocketMQ.
RocketMQ Flink: The Apache RocketMQ connector of Apache Flink that supports source and sink connector in data stream and Table.
RocketMQ APIs: RocketMQ protobuf protocol.
RocketMQ Clients: gRPC/protobuf-based RocketMQ clients.
RocketMQ Remoting-based Clients
RocketMQ Spring: A project which helps developers quickly integrate Apache RocketMQ with Spring Boot.
RocketMQ Exporter: An Apache RocketMQ exporter for Prometheus.
RocketMQ Operator: Providing a way to run an Apache RocketMQ cluster on Kubernetes.
RocketMQ Docker: The Git repo of the Docker Image for Apache RocketMQ.
RocketMQ Dashboard: Operation and maintenance console of Apache RocketMQ.
RocketMQ Connect: A tool for scalably and reliably streaming data between Apache RocketMQ and other systems.
RocketMQ MQTT: A new MQTT protocol architecture model, based on which Apache RocketMQ can better support messages from terminals such as IoT devices and Mobile APP.
RocketMQ EventBridge: EventBridge make it easier to build a event-driven application.
RocketMQ Incubating Community Projects: Incubator community projects of Apache RocketMQ, including logappender, rocketmq-ansible, rocketmq-beats-integration, rocketmq-cloudevents-binding, etc.
RocketMQ Site: The repository for Apache RocketMQ website.
RocketMQ E2E: A project for testing Apache RocketMQ, including end-to-end, performance, compatibility tests.

Learn it & Contact us

Mailing Lists: https://rocketmq.apache.org/about/contact/
Home: https://rocketmq.apache.org
Docs: https://rocketmq.apache.org/docs/quick-start/
Issues: https://github.com/apache/rocketmq/issues
Rips: https://github.com/apache/rocketmq/wiki/RocketMQ-Improvement-Proposal
Ask: https://stackoverflow.com/questions/tagged/rocketmq
Slack: https://rocketmq-invite-automation.herokuapp.com/

Contributing

We always welcome new contributions, whether for trivial cleanups, big new features or other material rewards, more details see here.

License

Apache License, Version 2.0 Copyright (C) Apache Software Foundation

Export Control Notice

This distribution includes cryptographic software. The country in which you currently reside may have
restrictions on the import, possession, use, and/or re-export to another country, of encryption software.
BEFORE using any encryption software, please check your country’s laws, regulations and policies concerning
the import, possession, or use, and re-export of encryption software, to see if this is permitted. See
http://www.wassenaar.org/ for more information.

The U.S. Government Department of Commerce, Bureau of Industry and Security (BIS), has classified this
software as Export Commodity Control Number (ECCN) 5D002.C.1, which includes information security software
using or performing cryptographic functions with asymmetric algorithms. The form and manner of this Apache
Software Foundation distribution makes it eligible for export under the License Exception ENC Technology
Software Unrestricted (TSU) exception (see the BIS Export Administration Regulations, Section 740.13) for
both object code and source code.

The following provides more details on the included cryptographic software:

This software uses Apache Commons Crypto (https://commons.apache.org/proper/commons-crypto/) to
support authentication, and encryption and decryption of data sent across the network between
services.

作者：Jeebiz 创建时间：2023-01-09 15:45
最后编辑：Jeebiz 更新时间：2024-09-23 10:03

1、RocketMQ 简介