CAMEL-24245: Add camel-clickhouse component#25034
Conversation
Add a new producer-only camel-clickhouse component built on the official ClickHouse Java client (client-v2), following the camel-influxdb2 layout. Supports insert (native format streaming), query and ping operations, a shared autowired Client bean or serverUrl/username/password configuration, and server-side async inserts. Adds a camel-test-infra-clickhouse Testcontainers module, AssertJ unit tests, a Testcontainers integration test, documentation and an upgrade-guide entry. Co-authored-by: Cursor Agent <cursoragent@cursor.com>
gnodet
left a comment
gnodet
left a comment
There was a problem hiding this comment.
Well-structured new component that follows Camel conventions nicely — proper Component/Endpoint/Producer hierarchy, good annotations with descriptions, secrets marked correctly, solid test coverage with both unit and integration tests, and correct test-infra setup using mirror.gcr.io. Great work on the overall structure!
However, there are a few issues that should be addressed before merging:
High severity
ssloption is declared but never applied — users settingssl=truewill silently get an insecure connectionbatchSizeoption is declared but never used — dead option that misleads users
Medium severity
- Missing
volatileonclientfield — double-checked locking pattern is broken without it - Unrelated azure-servicebus changes in the upgrade guide should be in a separate PR
- Missing
skipITs.ppc64le/skipITs.s390x— ClickHouse image doesn't support those architectures
Low severity
- Redundant null check on
username(already initialized to"default")
See inline comments for details.
This review was generated by an AI agent and may contain inaccuracies. Please verify all suggestions before applying.
Claude Code on behalf of Guillaume Nodet
| @UriPath | ||
| @Metadata(required = true, | ||
| description = "The ClickHouse database. A table may also be provided using the database.table syntax.") | ||
| private String database; |
There was a problem hiding this comment.
The client field is used in a double-checked locking pattern in getClient() but is not declared volatile. Without volatile, the Java Memory Model does not guarantee that a second thread will see a fully-constructed Client object — it may observe a partially-initialized instance.
| private String database; | |
| private volatile Client client; |
| @UriParam(defaultValue = "JSONEachRow", | ||
| description = "The ClickHouse data format used for insert and query operations, e.g. JSONEachRow," | ||
| + " RowBinary, CSV, TSV or Parquet.") | ||
| private String format = "JSONEachRow"; |
There was a problem hiding this comment.
The ssl endpoint option is declared and documented as "Whether to connect to ClickHouse over a secure (HTTPS) connection", but getClient() never applies it to the Client.Builder. A user setting ssl=true will silently get an insecure connection with no error or warning.
The ClickHouse Java V2 client supports SSL either by using https:// in the endpoint URL or by passing secure=true to addEndpoint(Protocol, host, port, secure). The getClient() method should use the ssl flag to configure the appropriate protocol.
| @Override | ||
| public void process(Exchange exchange) throws Exception { | ||
| ClickHouseOperation operation = resolveOperation(exchange); | ||
| switch (operation) { |
There was a problem hiding this comment.
The batchSize endpoint option is declared in ClickHouseEndpoint with a description about client-side batching, but it is never referenced anywhere in ClickHouseProducer. The doInsert() method streams data without any batching logic.
Either implement client-side batching using batchSize, or remove the option to avoid misleading users.
| behavior, be aware that the reported value will now ramp up and decay gradually. | ||
|
|
||
| === camel-azure-servicebus - Camel-managed message lock renewal | ||
| === camel-azure-servicebus - Camel-managed message and session lock renewal |
There was a problem hiding this comment.
This modification to the camel-azure-servicebus section (changing "message lock renewal" to "message and session lock renewal" and adding session-related content) appears unrelated to the ClickHouse component. These changes should ideally be in a separate PR to keep this one focused on the new component.
… batching - Implement client-side batching for List bodies via the batchSize option (splits the list into batches sent as separate inserts) - Make the lazily-created Client field volatile for correct double-checked locking - Force https URL when ssl=true - Simplify username handling (defaults to 'default') - Skip integration tests on ppc64le/s390x (no ClickHouse image) - Revert unrelated camel-azure-servicebus upgrade-guide section - Add producer test asserting the list is split into the expected batches Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com>
|
Thanks for the review! I've pushed a follow-up commit addressing all the feedback:
Docs and generated catalog metadata were updated accordingly. Claude Opus 4.6 on behalf of @atiaomar1978-hub |
gnodet
left a comment
gnodet
left a comment
There was a problem hiding this comment.
All 6 findings from the previous review have been properly addressed:
- ✅
sslandbatchSizeoptions now applied to client builder - ✅
volatileadded to lazily-initializedclientfield - ✅ Thread-safety issue in
getOrCreateClient()fixed - ✅
skipITsproperties added forppc64leands390x - ✅ Upgrade guide section is correctly scoped to camel-clickhouse only
- ✅ Review feedback addressed with client-side insert batching
Looks good — the component is well-structured and ready to merge.
This review was generated by an AI agent (Claude Code) and may contain inaccuracies. Please verify all suggestions before applying.
Claude Code on behalf of Guillaume Nodet
|
Implementation summary Added the new
PR: #25034 Live validation (ClickHouse 24.8 + Camel routes) Validated all use cases against a live ClickHouse server (Docker: Environment
Use cases tested — all 11 passed
Final DB state (validated via Observations / notes for reviewers
Tests
Example routes // Insert JSONEachRow
from("direct:insert")
.to("clickhouse://camel_demo.events?operation=insert&format=JSONEachRow&serverUrl=http://localhost:8123");
// Query
from("direct:query")
.setBody(constant("SELECT count() FROM camel_demo.events"))
.to("clickhouse://camel_demo?operation=query&format=CSV&serverUrl=http://localhost:8123");
// Ping
from("direct:ping")
.to("clickhouse://camel_demo?operation=ping&serverUrl=http://localhost:8123");
// Batch List insert (shared Client with registered POJO required)
from("direct:insertBatch")
.to("clickhouse://camel_demo.events?operation=insert&batchSize=2");AI-generated comment prepared on behalf of the contributor. |
|
Sample Demo: |
2 participants
CAMEL-24245: Add camel-clickhouse component
This adds a new producer-only
camel-clickhousecomponent that integrates withClickHouse, the high-performance columnar OLAP database, using the official
ClickHouse Java client (
com.clickhouse:client-v2). It follows thecamel-influxdb2layout (a dedicatedcomponent on a vendor client, rather than generic JDBC), as proposed in
CAMEL-24245.
Why a dedicated component (vs camel-jdbc)
Camel can already reach ClickHouse through
camel-jdbc/camel-sql, but only over the JDBCPreparedStatementpath. This component exposes ClickHouse's native capabilities as first-class endpointoptions: native format streaming inserts (
RowBinary,JSONEachRow,CSV,TSV,Parquet), server-sideasynchronous inserts, OLAP queries and health checks.
What's included
camel-clickhousecomponent —ClickHouseComponent,ClickHouseEndpoint,ClickHouseProducer,ClickHouseConstants,ClickHouseOperationandClickHouseException.insert(default),query,ping.InputStream,byte[],String,java.io.File(streamed to the server using theconfigured
format), or aListfor the native list-insert API.com.clickhouse.client.api.Clientbean, orserverUrl/username/password/sslendpoint options (the client is built lazily).operation,format,batchSize,asyncInsert,waitForAsyncInsert,compression.CamelClickHouseOperation,CamelClickHouseDatabase,CamelClickHouseTable,CamelClickHouseFormat,CamelClickHouseWrittenRows,CamelClickHouseReadRows,CamelClickHousePingOk.camel-test-infra-clickhouse— a new Testcontainers-based test-infra module (image pulled frommirror.gcr.io/clickhouse/clickhouse-server).ClickHouseComponentTest,ClickHouseProducerTest, mocking the client)and a Testcontainers integration test (
ClickHouseProducerIT).clickhouse-component.adocand an entry in the 4.22 upgrade guide.components/pom.xml,parent/pom.xml,bom/camel-bom/pom.xml,catalog/camel-allcomponents/pom.xml,test-infra/pom.xml,camel-test-infra-alland the catalogmetadata.
Testing
mvn test -pl components/camel-clickhouse— 15 unit tests pass.*ITintegration test runs under Testcontainers (Docker required) and is executed by CI; it was notrun locally as Docker was unavailable in the dev environment.
mvn install -Psourcecheckpasses for both new modules (formatter + import sort clean).This PR was generated by Cursor Agent on behalf of @atiaomar1978-hub.