Copyright 2025, Altinity Inc.. All Rights Reserved. All information contained herein is, and remains the property of Altinity Inc.. Any dissemination of this information or reproduction of this material is strictly forbidden unless prior written permission is obtained from Altinity Inc..

Authentication Test Run Report

DateJul 11, 2025 0:08
Duration35m 13s
Framework TestFlows 2.0.250110.1002922

Artifacts

Test artifacts can be found at https://altinity-build-artifacts.s3.amazonaws.com/index.html#0/1abef611c9a40dce30ed011ede491e532b2a3e72/regression/aarch64/with_analyzer/zookeeper/without_thread_fuzzer/ldap/authentication/

Attributes

projectAltinity/ClickHouse
project.id159717931
packagehttps://s3.amazonaws.com/altinity-build-artifacts/25.3/1abef611c9a40dce30ed011ede491e532b2a3e72/package_aarch64/clickhouse-common-static_25.3.3.20183.altinityantalya_arm64.deb
version25.3.3.20183.altinityantalya
user.nameEnmk
repositoryhttps://github.com/Altinity/clickhouse-regression
commit.hash88c93f843cd48cd9defc6cec6b98d6b98f94adde
job.nameLDAP (authentication)
job.retry1
job.urlhttps://github.com/Altinity/ClickHouse/actions/runs/16207084940
archaarch64
localTrue
clickhouse_versionNone
clickhouse_pathhttps://s3.amazonaws.com/altinity-build-artifacts/25.3/1abef611c9a40dce30ed011ede491e532b2a3e72/package_aarch64/clickhouse-common-static_25.3.3.20183.altinityantalya_arm64.deb
as_binaryFalse
base_osNone
keeper_pathNone
zookeeper_versionNone
use_keeperFalse
stressFalse
collect_service_logsTrue
thread_fuzzerFalse
with_analyzerTrue
reuse_envFalse

Summary

93.7%OK
6.3%Known

Statistics

Units OK Fail Error XFail
Features
8
8
Scenarios
93
89
4
Examples
25
21
4
Steps
6328
6306
16
6

Known Fails

Test NameResultMessage
/authentication/connection protocols/tls cipher suiteXFail 22s 286ms
can't get it to work
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 110, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/argparser.py", line 171, in capture_cluster_args
    return func(self, cluster_args=cluster_args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 102, in regression
    Feature(run=load("ldap.authentication.tests.connections", "feature"))
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/connections.py", line 311, in feature
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/connections.py", line 242, in tls_cipher_suite
    login(servers, *users)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/common.py", line 438, in login
    current().context.node.query(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py", line 1188, in query
    assert False, error(r.output)
           ^^^^^
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert False, error(r.output)

Description
  Code: 516. DB::Exception: Received from localhost:9000. DB::Exception: user4: Authentication failed: password is incorrect, or there is no user with such name.. (AUTHENTICATION_FAILED)

Assertion values
  assert False, error(r.output)
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py', line 1188 in 'query'

1180\|                  assert message in r.output, error(r.output)
1181\|  
1182\|          if not ignore_exception:
1183\|              if message is None or "Exception:" not in message:
1184\|                  with Then("check if output has exception") if steps else NullStep():
1185\|                      if "Exception:" in r.output:
1186\|                          if raise_on_exception:
1187\|                              raise QueryRuntimeException(r.output)
1188\|>                         assert False, error(r.output)
1189\|  
1190\|          return r
1191\|
/authentication/connection protocols/tls require cert default demandXFail 22s 419ms
can't be tested with self-signed certificates
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 110, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/argparser.py", line 171, in capture_cluster_args
    return func(self, cluster_args=cluster_args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 102, in regression
    Feature(run=load("ldap.authentication.tests.connections", "feature"))
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/connections.py", line 311, in feature
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/connections.py", line 199, in tls_require_cert_default_demand
    login(servers, *users)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/common.py", line 438, in login
    current().context.node.query(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py", line 1188, in query
    assert False, error(r.output)
           ^^^^^
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert False, error(r.output)

Description
  Code: 516. DB::Exception: Received from localhost:9000. DB::Exception: user2: Authentication failed: password is incorrect, or there is no user with such name.. (AUTHENTICATION_FAILED)

Assertion values
  assert False, error(r.output)
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py', line 1188 in 'query'

1180\|                  assert message in r.output, error(r.output)
1181\|  
1182\|          if not ignore_exception:
1183\|              if message is None or "Exception:" not in message:
1184\|                  with Then("check if output has exception") if steps else NullStep():
1185\|                      if "Exception:" in r.output:
1186\|                          if raise_on_exception:
1187\|                              raise QueryRuntimeException(r.output)
1188\|>                         assert False, error(r.output)
1189\|  
1190\|          return r
1191\|
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown reset when invalid password is providedXFail 22s 778ms
https://github.com/ClickHouse/ClickHouse/issues/51323
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 110, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/argparser.py", line 171, in capture_cluster_args
    return func(self, cluster_args=cluster_args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 105, in regression
    Feature(run=load("ldap.authentication.tests.authentications", "feature"))
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/authentications.py", line 1228, in feature
    Feature(test=verification_cooldown)(rbac=rbac, servers=servers, node=node)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/authentications.py", line 1203, in verification_cooldown
    scenario(server="openldap1", rbac=rbac)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/authentications.py", line 1184, in scenario
    login_and_execute_query(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/authentications.py", line 36, in login_and_execute_query
    self.context.node.query(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py", line 1172, in query
    assert r.exitcode == exitcode, error(r.output)
           ^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert r.exitcode == exitcode, error(r.output)

Description
  1

Assertion values
  assert r.exitcode == exitcode, error(r.output)
         ^ is 
  assert r.exitcode == exitcode, error(r.output)
         ^ is = 0
  assert r.exitcode == exitcode, error(r.output)
                       ^ is 4
  assert r.exitcode == exitcode, error(r.output)
                    ^ is = False
  assert r.exitcode == exitcode, error(r.output)
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py', line 1172 in 'query'

1164\|                      **kwargs,
1165\|                  )
1166\|  
1167\|          if no_checks:
1168\|              return r
1169\|  
1170\|          if exitcode is not None:
1171\|              with Then(f"exitcode should be {exitcode}") if steps else NullStep():
1172\|>                 assert r.exitcode == exitcode, error(r.output)
1173\|  
1174\|          if message is not None:
1175\|              with (
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown reset when invalid password is providedXFail 22s 559ms
https://github.com/ClickHouse/ClickHouse/issues/51323
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 110, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/argparser.py", line 171, in capture_cluster_args
    return func(self, cluster_args=cluster_args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 105, in regression
    Feature(run=load("ldap.authentication.tests.authentications", "feature"))
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/authentications.py", line 1228, in feature
    Feature(test=verification_cooldown)(rbac=rbac, servers=servers, node=node)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/authentications.py", line 1203, in verification_cooldown
    scenario(server="openldap1", rbac=rbac)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/authentications.py", line 1184, in scenario
    login_and_execute_query(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/authentications.py", line 36, in login_and_execute_query
    self.context.node.query(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py", line 1172, in query
    assert r.exitcode == exitcode, error(r.output)
           ^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert r.exitcode == exitcode, error(r.output)

Description
  1

Assertion values
  assert r.exitcode == exitcode, error(r.output)
         ^ is 
  assert r.exitcode == exitcode, error(r.output)
         ^ is = 0
  assert r.exitcode == exitcode, error(r.output)
                       ^ is 4
  assert r.exitcode == exitcode, error(r.output)
                    ^ is = False
  assert r.exitcode == exitcode, error(r.output)
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py', line 1172 in 'query'

1164\|                      **kwargs,
1165\|                  )
1166\|  
1167\|          if no_checks:
1168\|              return r
1169\|  
1170\|          if exitcode is not None:
1171\|              with Then(f"exitcode should be {exitcode}") if steps else NullStep():
1172\|>                 assert r.exitcode == exitcode, error(r.output)
1173\|  
1174\|          if message is not None:
1175\|              with (

Results

Test Name Result Duration
/authentication OK 35m 13s
/authentication/sanity OK 1s 611ms
/authentication/multiple servers OK 21s 449ms
/authentication/connection protocols OK 7m 26s
/authentication/connection protocols/plain text OK 22s 219ms
/authentication/connection protocols/plain text with custom port OK 22s 236ms
/authentication/connection protocols/starttls OK 1m 29s
/authentication/connection protocols/starttls with custom port OK 22s 327ms
/authentication/connection protocols/tls OK 1m 29s
/authentication/connection protocols/tls cipher suite XFail 22s 286ms
/authentication/connection protocols/tls enable tls default yes OK 22s 310ms
/authentication/connection protocols/tls minimum protocol version OK 1m 52s
/authentication/connection protocols/tls require cert default demand XFail 22s 419ms
/authentication/connection protocols/tls with custom port OK 22s 492ms
/authentication/server config OK 4m 30s
/authentication/server config/auth dn value OK 22s 526ms
/authentication/server config/empty ca cert dir OK 22s 387ms
/authentication/server config/empty ca cert file OK 22s 424ms
/authentication/server config/empty host OK 22s 382ms
/authentication/server config/empty server name OK 2s 100ms
/authentication/server config/invalid auth dn prefix OK 24s 385ms
/authentication/server config/invalid auth dn suffix OK 22s 523ms
/authentication/server config/invalid enable tls value OK 22s 367ms
/authentication/server config/invalid host OK 22s 464ms
/authentication/server config/invalid port OK 22s 536ms
/authentication/server config/invalid tls require cert value OK 22s 609ms
/authentication/server config/invalid verification cooldown value OK 12s 569ms
/authentication/server config/missing host OK 23s 443ms
/authentication/server config/syntax OK 6s 130ms
/authentication/user config OK 1m 30s
/authentication/user config/empty server name OK 8s 103ms
/authentication/user config/empty server not defined OK 22s 478ms
/authentication/user config/empty user name OK 6s 162ms
/authentication/user config/ldap and password OK 8s 178ms
/authentication/user config/multiple servers OK 22s 549ms
/authentication/user config/valid user config OK 22s 620ms
/authentication/user authentications OK 19m 13s
/authentication/user authentications/rbac=False OK 12m 12s
/authentication/user authentications/rbac=False/add user to ldap and login OK 16s 507ms
/authentication/user authentications/rbac=False/empty username and empty password OK 83ms
/authentication/user authentications/rbac=False/invalid long username and valid short password OK 16s 501ms
/authentication/user authentications/rbac=False/invalid username and valid password OK 16s 525ms
/authentication/user authentications/rbac=False/login after clickhouse server is restarted OK 30s 481ms
/authentication/user authentications/rbac=False/login after ldap server is restarted OK 19s 633ms
/authentication/user authentications/rbac=False/login after user cn changed in ldap OK 16s 428ms
/authentication/user authentications/rbac=False/login after user is deleted from ldap OK 16s 301ms
/authentication/user authentications/rbac=False/login after user password changed in ldap OK 16s 468ms
/authentication/user authentications/rbac=False/parallel login OK 35s 632ms
/authentication/user authentications/rbac=False/valid ascii username and utf8 password OK 16s 321ms
/authentication/user authentications/rbac=False/valid long username and short password OK 16s 360ms
/authentication/user authentications/rbac=False/valid short username and invalid long password OK 16s 342ms
/authentication/user authentications/rbac=False/valid short username and long password OK 16s 327ms
/authentication/user authentications/rbac=False/valid username and invalid empty password OK 16s 353ms
/authentication/user authentications/rbac=False/valid username and invalid password OK 16s 367ms
/authentication/user authentications/rbac=False/valid username and password OK 16s 351ms
/authentication/user authentications/rbac=False/valid username and password invalid server OK 16s 217ms
/authentication/user authentications/rbac=False/valid username with valid empty password OK 16s 427ms
/authentication/user authentications/rbac=False/valid utf8 username and ascii password OK 16s 358ms
/authentication/user authentications/rbac=False/verification cooldown OK 6m 18s
/authentication/user authentications/rbac=False/verification cooldown/default verification cooldown value OK 22s 558ms
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown reset when invalid password is provided XFail 22s 778ms
/authentication/user authentications/rbac=False/verification cooldown/valid verification cooldown value cn change OK 22s 567ms
/authentication/user authentications/rbac=False/verification cooldown/valid verification cooldown value ldap unavailable OK 28s 100ms
/authentication/user authentications/rbac=False/verification cooldown/valid verification cooldown value password change OK 22s 667ms
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown performance OK 1m 55s
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown reset on server auth dn prefix parameter change OK 36s 756ms
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown reset on server auth dn suffix parameter change OK 35s 622ms
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown reset on server host parameter change OK 35s 727ms
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown reset on server port parameter change OK 35s 773ms
/authentication/user authentications/rbac=True OK 7m 1s
/authentication/user authentications/rbac=True/add user to ldap and login OK 290ms
/authentication/user authentications/rbac=True/empty username and empty password OK 80ms
/authentication/user authentications/rbac=True/invalid long username and valid short password OK 279ms
/authentication/user authentications/rbac=True/invalid username and valid password OK 275ms
/authentication/user authentications/rbac=True/login after clickhouse server is restarted OK 14s 300ms
/authentication/user authentications/rbac=True/login after ldap server is restarted OK 3s 630ms
/authentication/user authentications/rbac=True/login after user cn changed in ldap OK 370ms
/authentication/user authentications/rbac=True/login after user is deleted from ldap OK 351ms
/authentication/user authentications/rbac=True/login after user password changed in ldap OK 441ms
/authentication/user authentications/rbac=True/parallel login OK 19s 418ms
/authentication/user authentications/rbac=True/valid ascii username and utf8 password OK 314ms
/authentication/user authentications/rbac=True/valid long username and short password OK 290ms
/authentication/user authentications/rbac=True/valid short username and invalid long password OK 294ms
/authentication/user authentications/rbac=True/valid short username and long password OK 262ms
/authentication/user authentications/rbac=True/valid username and invalid empty password OK 252ms
/authentication/user authentications/rbac=True/valid username and invalid password OK 260ms
/authentication/user authentications/rbac=True/valid username and password OK 278ms
/authentication/user authentications/rbac=True/valid username and password invalid server OK 174ms
/authentication/user authentications/rbac=True/valid username with valid empty password OK 250ms
/authentication/user authentications/rbac=True/valid utf8 username and ascii password OK 267ms
/authentication/user authentications/rbac=True/verification cooldown OK 6m 13s
/authentication/user authentications/rbac=True/verification cooldown/default verification cooldown value OK 22s 401ms
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown reset when invalid password is provided XFail 22s 559ms
/authentication/user authentications/rbac=True/verification cooldown/valid verification cooldown value cn change OK 22s 446ms
/authentication/user authentications/rbac=True/verification cooldown/valid verification cooldown value ldap unavailable OK 27s 989ms
/authentication/user authentications/rbac=True/verification cooldown/valid verification cooldown value password change OK 22s 460ms
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown performance OK 1m 52s
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown reset on server auth dn prefix parameter change OK 36s 405ms
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown reset on server auth dn suffix parameter change OK 35s 534ms
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown reset on server host parameter change OK 35s 598ms
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown reset on server port parameter change OK 35s 716ms
/authentication/multiple authentications methods OK 6s 697ms
/authentication/multiple authentications methods/ldap with other auth methods OK 1s 646ms

Generated by TestFlows Open-Source Test Framework v2.0.250110.1002922