Copyright 2025, Altinity Inc.. All Rights Reserved. All information contained herein is, and remains the property of Altinity Inc.. Any dissemination of this information or reproduction of this material is strictly forbidden unless prior written permission is obtained from Altinity Inc..

Authentication Test Run Report

DateJul 28, 2025 9:45
Duration31m 39s
Framework TestFlows 2.0.250110.1002922

Artifacts

Test artifacts can be found at https://altinity-build-artifacts.s3.amazonaws.com/index.html#0/d32d0074004db61e346611c777e26532a456fe2f/regression/aarch64/with_analyzer/zookeeper/without_thread_fuzzer/ldap/authentication/

Attributes

projectAltinity/ClickHouse
project.id159717931
packagehttps://s3.amazonaws.com/altinity-build-artifacts/25.3/d32d0074004db61e346611c777e26532a456fe2f/package_aarch64/clickhouse-common-static_25.3.6.10034.altinitystable_arm64.deb
version25.3.6.10034.altinitystable
user.namezvonand
repositoryhttps://github.com/Altinity/clickhouse-regression
commit.hash5723e20cbc49b347114c7b90c7316a44dafa5328
job.nameLDAP (authentication)
job.retry1
job.urlhttps://github.com/Altinity/ClickHouse/actions/runs/16564472800
archaarch64
localTrue
clickhouse_versionNone
clickhouse_pathhttps://s3.amazonaws.com/altinity-build-artifacts/25.3/d32d0074004db61e346611c777e26532a456fe2f/package_aarch64/clickhouse-common-static_25.3.6.10034.altinitystable_arm64.deb
as_binaryFalse
base_osNone
keeper_pathNone
zookeeper_versionNone
use_keeperFalse
stressFalse
collect_service_logsTrue
thread_fuzzerFalse
with_analyzerTrue
reuse_envFalse

Summary

93.7%OK
6.3%Known

Statistics

Units OK Fail XFail
Features
8
8
Scenarios
93
89
4
Examples
25
21
4
Steps
6336
6320
16

Known Fails

Test NameResultMessage
/authentication/connection protocols/tls cipher suiteXFail 20s 341ms
can't get it to work
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 110, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/argparser.py", line 171, in capture_cluster_args
    return func(self, cluster_args=cluster_args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 102, in regression
    Feature(run=load("ldap.authentication.tests.connections", "feature"))
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/connections.py", line 311, in feature
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/connections.py", line 242, in tls_cipher_suite
    login(servers, *users)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/common.py", line 438, in login
    current().context.node.query(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py", line 1195, in query
    assert False, error(r.output)
           ^^^^^
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert False, error(r.output)

Description
  Code: 516. DB::Exception: Received from localhost:9000. DB::Exception: user4: Authentication failed: password is incorrect, or there is no user with such name.. (AUTHENTICATION_FAILED)

Assertion values
  assert False, error(r.output)
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py', line 1195 in 'query'

1187\|                  assert message in r.output, error(r.output)
1188\|  
1189\|          if not ignore_exception:
1190\|              if message is None or "Exception:" not in message:
1191\|                  with Then("check if output has exception") if steps else NullStep():
1192\|                      if "Exception:" in r.output:
1193\|                          if raise_on_exception:
1194\|                              raise QueryRuntimeException(r.output)
1195\|>                         assert False, error(r.output)
1196\|  
1197\|          return r
1198\|
/authentication/connection protocols/tls require cert default demandXFail 20s 449ms
can't be tested with self-signed certificates
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 110, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/argparser.py", line 171, in capture_cluster_args
    return func(self, cluster_args=cluster_args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 102, in regression
    Feature(run=load("ldap.authentication.tests.connections", "feature"))
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/connections.py", line 311, in feature
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/connections.py", line 199, in tls_require_cert_default_demand
    login(servers, *users)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/common.py", line 438, in login
    current().context.node.query(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py", line 1195, in query
    assert False, error(r.output)
           ^^^^^
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert False, error(r.output)

Description
  Code: 516. DB::Exception: Received from localhost:9000. DB::Exception: user2: Authentication failed: password is incorrect, or there is no user with such name.. (AUTHENTICATION_FAILED)

Assertion values
  assert False, error(r.output)
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py', line 1195 in 'query'

1187\|                  assert message in r.output, error(r.output)
1188\|  
1189\|          if not ignore_exception:
1190\|              if message is None or "Exception:" not in message:
1191\|                  with Then("check if output has exception") if steps else NullStep():
1192\|                      if "Exception:" in r.output:
1193\|                          if raise_on_exception:
1194\|                              raise QueryRuntimeException(r.output)
1195\|>                         assert False, error(r.output)
1196\|  
1197\|          return r
1198\|
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown reset when invalid password is providedXFail 19s 850ms
https://github.com/ClickHouse/ClickHouse/issues/51323
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 110, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/argparser.py", line 171, in capture_cluster_args
    return func(self, cluster_args=cluster_args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 105, in regression
    Feature(run=load("ldap.authentication.tests.authentications", "feature"))
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/authentications.py", line 1228, in feature
    Feature(test=verification_cooldown)(rbac=rbac, servers=servers, node=node)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/authentications.py", line 1203, in verification_cooldown
    scenario(server="openldap1", rbac=rbac)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/authentications.py", line 1184, in scenario
    login_and_execute_query(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/authentications.py", line 36, in login_and_execute_query
    self.context.node.query(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py", line 1179, in query
    assert r.exitcode == exitcode, error(r.output)
           ^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert r.exitcode == exitcode, error(r.output)

Description
  1

Assertion values
  assert r.exitcode == exitcode, error(r.output)
         ^ is 
  assert r.exitcode == exitcode, error(r.output)
         ^ is = 0
  assert r.exitcode == exitcode, error(r.output)
                       ^ is 4
  assert r.exitcode == exitcode, error(r.output)
                    ^ is = False
  assert r.exitcode == exitcode, error(r.output)
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py', line 1179 in 'query'

1171\|                      **kwargs,
1172\|                  )
1173\|  
1174\|          if no_checks:
1175\|              return r
1176\|  
1177\|          if exitcode is not None:
1178\|              with Then(f"exitcode should be {exitcode}") if steps else NullStep():
1179\|>                 assert r.exitcode == exitcode, error(r.output)
1180\|  
1181\|          if message is not None:
1182\|              with (
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown reset when invalid password is providedXFail 19s 601ms
https://github.com/ClickHouse/ClickHouse/issues/51323
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 110, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/argparser.py", line 171, in capture_cluster_args
    return func(self, cluster_args=cluster_args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 105, in regression
    Feature(run=load("ldap.authentication.tests.authentications", "feature"))
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/authentications.py", line 1228, in feature
    Feature(test=verification_cooldown)(rbac=rbac, servers=servers, node=node)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/authentications.py", line 1203, in verification_cooldown
    scenario(server="openldap1", rbac=rbac)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/authentications.py", line 1184, in scenario
    login_and_execute_query(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/authentications.py", line 36, in login_and_execute_query
    self.context.node.query(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py", line 1179, in query
    assert r.exitcode == exitcode, error(r.output)
           ^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert r.exitcode == exitcode, error(r.output)

Description
  1

Assertion values
  assert r.exitcode == exitcode, error(r.output)
         ^ is 
  assert r.exitcode == exitcode, error(r.output)
         ^ is = 0
  assert r.exitcode == exitcode, error(r.output)
                       ^ is 4
  assert r.exitcode == exitcode, error(r.output)
                    ^ is = False
  assert r.exitcode == exitcode, error(r.output)
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py', line 1179 in 'query'

1171\|                      **kwargs,
1172\|                  )
1173\|  
1174\|          if no_checks:
1175\|              return r
1176\|  
1177\|          if exitcode is not None:
1178\|              with Then(f"exitcode should be {exitcode}") if steps else NullStep():
1179\|>                 assert r.exitcode == exitcode, error(r.output)
1180\|  
1181\|          if message is not None:
1182\|              with (

Results

Test Name Result Duration
/authentication OK 31m 39s
/authentication/sanity OK 328ms
/authentication/multiple servers OK 20s 440ms
/authentication/connection protocols OK 6m 47s
/authentication/connection protocols/plain text OK 20s 286ms
/authentication/connection protocols/plain text with custom port OK 20s 222ms
/authentication/connection protocols/starttls OK 1m 21s
/authentication/connection protocols/starttls with custom port OK 20s 430ms
/authentication/connection protocols/tls OK 1m 21s
/authentication/connection protocols/tls cipher suite XFail 20s 341ms
/authentication/connection protocols/tls enable tls default yes OK 20s 433ms
/authentication/connection protocols/tls minimum protocol version OK 1m 42s
/authentication/connection protocols/tls require cert default demand XFail 20s 449ms
/authentication/connection protocols/tls with custom port OK 20s 420ms
/authentication/server config OK 4m 3s
/authentication/server config/auth dn value OK 20s 435ms
/authentication/server config/empty ca cert dir OK 20s 492ms
/authentication/server config/empty ca cert file OK 20s 414ms
/authentication/server config/empty host OK 20s 538ms
/authentication/server config/empty server name OK 2s 103ms
/authentication/server config/invalid auth dn prefix OK 20s 483ms
/authentication/server config/invalid auth dn suffix OK 20s 432ms
/authentication/server config/invalid enable tls value OK 20s 488ms
/authentication/server config/invalid host OK 20s 459ms
/authentication/server config/invalid port OK 20s 497ms
/authentication/server config/invalid tls require cert value OK 20s 503ms
/authentication/server config/invalid verification cooldown value OK 12s 622ms
/authentication/server config/missing host OK 19s 490ms
/authentication/server config/syntax OK 4s 588ms
/authentication/user config OK 1m 18s
/authentication/user config/empty server name OK 5s 641ms
/authentication/user config/empty server not defined OK 20s 497ms
/authentication/user config/empty user name OK 6s 673ms
/authentication/user config/ldap and password OK 5s 689ms
/authentication/user config/multiple servers OK 19s 471ms
/authentication/user config/valid user config OK 20s 603ms
/authentication/user authentications OK 17m 22s
/authentication/user authentications/rbac=False OK 11m 12s
/authentication/user authentications/rbac=False/add user to ldap and login OK 15s 142ms
/authentication/user authentications/rbac=False/empty username and empty password OK 93ms
/authentication/user authentications/rbac=False/invalid long username and valid short password OK 16s 109ms
/authentication/user authentications/rbac=False/invalid username and valid password OK 16s 73ms
/authentication/user authentications/rbac=False/login after clickhouse server is restarted OK 27s 552ms
/authentication/user authentications/rbac=False/login after ldap server is restarted OK 15s 691ms
/authentication/user authentications/rbac=False/login after user cn changed in ldap OK 15s 873ms
/authentication/user authentications/rbac=False/login after user is deleted from ldap OK 15s 976ms
/authentication/user authentications/rbac=False/login after user password changed in ldap OK 15s 974ms
/authentication/user authentications/rbac=False/parallel login OK 29s 15ms
/authentication/user authentications/rbac=False/valid ascii username and utf8 password OK 15s 871ms
/authentication/user authentications/rbac=False/valid long username and short password OK 15s 912ms
/authentication/user authentications/rbac=False/valid short username and invalid long password OK 15s 795ms
/authentication/user authentications/rbac=False/valid short username and long password OK 15s 849ms
/authentication/user authentications/rbac=False/valid username and invalid empty password OK 15s 885ms
/authentication/user authentications/rbac=False/valid username and invalid password OK 15s 837ms
/authentication/user authentications/rbac=False/valid username and password OK 15s 861ms
/authentication/user authentications/rbac=False/valid username and password invalid server OK 15s 731ms
/authentication/user authentications/rbac=False/valid username with valid empty password OK 15s 948ms
/authentication/user authentications/rbac=False/valid utf8 username and ascii password OK 15s 906ms
/authentication/user authentications/rbac=False/verification cooldown OK 5m 42s
/authentication/user authentications/rbac=False/verification cooldown/default verification cooldown value OK 20s 628ms
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown reset when invalid password is provided XFail 19s 850ms
/authentication/user authentications/rbac=False/verification cooldown/valid verification cooldown value cn change OK 19s 611ms
/authentication/user authentications/rbac=False/verification cooldown/valid verification cooldown value ldap unavailable OK 21s 582ms
/authentication/user authentications/rbac=False/verification cooldown/valid verification cooldown value password change OK 19s 620ms
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown performance OK 1m 50s
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown reset on server auth dn prefix parameter change OK 32s 464ms
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown reset on server auth dn suffix parameter change OK 32s 525ms
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown reset on server host parameter change OK 32s 460ms
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown reset on server port parameter change OK 32s 548ms
/authentication/user authentications/rbac=True OK 6m 9s
/authentication/user authentications/rbac=True/add user to ldap and login OK 300ms
/authentication/user authentications/rbac=True/empty username and empty password OK 79ms
/authentication/user authentications/rbac=True/invalid long username and valid short password OK 298ms
/authentication/user authentications/rbac=True/invalid username and valid password OK 258ms
/authentication/user authentications/rbac=True/login after clickhouse server is restarted OK 11s 739ms
/authentication/user authentications/rbac=True/login after ldap server is restarted OK 1s 198ms
/authentication/user authentications/rbac=True/login after user cn changed in ldap OK 381ms
/authentication/user authentications/rbac=True/login after user is deleted from ldap OK 337ms
/authentication/user authentications/rbac=True/login after user password changed in ldap OK 425ms
/authentication/user authentications/rbac=True/parallel login OK 14s 189ms
/authentication/user authentications/rbac=True/valid ascii username and utf8 password OK 270ms
/authentication/user authentications/rbac=True/valid long username and short password OK 266ms
/authentication/user authentications/rbac=True/valid short username and invalid long password OK 238ms
/authentication/user authentications/rbac=True/valid short username and long password OK 279ms
/authentication/user authentications/rbac=True/valid username and invalid empty password OK 237ms
/authentication/user authentications/rbac=True/valid username and invalid password OK 234ms
/authentication/user authentications/rbac=True/valid username and password OK 275ms
/authentication/user authentications/rbac=True/valid username and password invalid server OK 165ms
/authentication/user authentications/rbac=True/valid username with valid empty password OK 237ms
/authentication/user authentications/rbac=True/valid utf8 username and ascii password OK 274ms
/authentication/user authentications/rbac=True/verification cooldown OK 5m 33s
/authentication/user authentications/rbac=True/verification cooldown/default verification cooldown value OK 19s 443ms
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown reset when invalid password is provided XFail 19s 601ms
/authentication/user authentications/rbac=True/verification cooldown/valid verification cooldown value cn change OK 19s 527ms
/authentication/user authentications/rbac=True/verification cooldown/valid verification cooldown value ldap unavailable OK 19s 284ms
/authentication/user authentications/rbac=True/verification cooldown/valid verification cooldown value password change OK 19s 473ms
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown performance OK 1m 47s
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown reset on server auth dn prefix parameter change OK 32s 234ms
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown reset on server auth dn suffix parameter change OK 32s 399ms
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown reset on server host parameter change OK 32s 352ms
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown reset on server port parameter change OK 32s 336ms
/authentication/multiple authentications methods OK 6s 188ms
/authentication/multiple authentications methods/ldap with other auth methods OK 1s 617ms

Generated by TestFlows Open-Source Test Framework v2.0.250110.1002922