Copyright 2025, Altinity Inc.. All Rights Reserved. All information contained herein is, and remains the property of Altinity Inc.. Any dissemination of this information or reproduction of this material is strictly forbidden unless prior written permission is obtained from Altinity Inc..

Authentication Test Run Report

DateApr 01, 2025 22:12
Duration35m 28s
Framework TestFlows 2.0.250110.1002922

Artifacts

Test artifacts can be found at https://altinity-build-artifacts.s3.amazonaws.com/index.html#708/12fb72fa077bd3f529f48fabb290d280d46348de/regression/aarch64/with_analyzer/zookeeper/without_thread_fuzzer/ldap/authentication/

Attributes

projectAltinity/ClickHouse
project.id159717931
packagehttps://s3.amazonaws.com/altinity-build-artifacts/PRs/708/12fb72fa077bd3f529f48fabb290d280d46348de/package_aarch64/clickhouse-common-static_24.12.2.20221.altinityantalya_arm64.deb
version24.12.2.20221.altinityantalya
user.nameianton-ru
repositoryhttps://github.com/Altinity/clickhouse-regression
commit.hashbd31e738c0cedaca253d15a05ed245c41b6e0b6a
job.nameLDAP
job.retry1
job.urlhttps://github.com/Altinity/ClickHouse/actions/runs/14205957338
archaarch64
localTrue
clickhouse_versionNone
clickhouse_pathhttps://s3.amazonaws.com/altinity-build-artifacts/PRs/708/12fb72fa077bd3f529f48fabb290d280d46348de/package_aarch64/clickhouse-common-static_24.12.2.20221.altinityantalya_arm64.deb
as_binaryFalse
base_osNone
keeper_pathNone
zookeeper_versionNone
use_keeperFalse
stressFalse
collect_service_logsTrue
thread_fuzzerFalse
with_analyzerTrue
reuse_envFalse

Summary

93.7%OK
6.3%Known

Statistics

Units OK Fail Error XFail
Features
8
8
Scenarios
93
89
4
Examples
25
21
4
Steps
6327
6305
16
6

Known Fails

Test NameResultMessage
/authentication/connection protocols/tls cipher suiteXFail 21s 902ms
can't get it to work
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 110, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/argparser.py", line 171, in capture_cluster_args
    return func(self, cluster_args=cluster_args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 102, in regression
    Feature(run=load("ldap.authentication.tests.connections", "feature"))
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/connections.py", line 311, in feature
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/connections.py", line 242, in tls_cipher_suite
    login(servers, *users)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/common.py", line 438, in login
    current().context.node.query(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py", line 1188, in query
    assert False, error(r.output)
           ^^^^^
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert False, error(r.output)

Description
  Code: 516. DB::Exception: Received from localhost:9000. DB::Exception: user4: Authentication failed: password is incorrect, or there is no user with such name.. (AUTHENTICATION_FAILED)

Assertion values
  assert False, error(r.output)
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py', line 1188 in 'query'

1180\|                  assert message in r.output, error(r.output)
1181\|  
1182\|          if not ignore_exception:
1183\|              if message is None or "Exception:" not in message:
1184\|                  with Then("check if output has exception") if steps else NullStep():
1185\|                      if "Exception:" in r.output:
1186\|                          if raise_on_exception:
1187\|                              raise QueryRuntimeException(r.output)
1188\|>                         assert False, error(r.output)
1189\|  
1190\|          return r
1191\|
/authentication/connection protocols/tls require cert default demandXFail 21s 872ms
can't be tested with self-signed certificates
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 110, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/argparser.py", line 171, in capture_cluster_args
    return func(self, cluster_args=cluster_args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 102, in regression
    Feature(run=load("ldap.authentication.tests.connections", "feature"))
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/connections.py", line 311, in feature
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/connections.py", line 199, in tls_require_cert_default_demand
    login(servers, *users)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/common.py", line 438, in login
    current().context.node.query(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py", line 1188, in query
    assert False, error(r.output)
           ^^^^^
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert False, error(r.output)

Description
  Code: 516. DB::Exception: Received from localhost:9000. DB::Exception: user2: Authentication failed: password is incorrect, or there is no user with such name.. (AUTHENTICATION_FAILED)

Assertion values
  assert False, error(r.output)
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py', line 1188 in 'query'

1180\|                  assert message in r.output, error(r.output)
1181\|  
1182\|          if not ignore_exception:
1183\|              if message is None or "Exception:" not in message:
1184\|                  with Then("check if output has exception") if steps else NullStep():
1185\|                      if "Exception:" in r.output:
1186\|                          if raise_on_exception:
1187\|                              raise QueryRuntimeException(r.output)
1188\|>                         assert False, error(r.output)
1189\|  
1190\|          return r
1191\|
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown reset when invalid password is providedXFail 22s 216ms
https://github.com/ClickHouse/ClickHouse/issues/51323
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 110, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/argparser.py", line 171, in capture_cluster_args
    return func(self, cluster_args=cluster_args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 105, in regression
    Feature(run=load("ldap.authentication.tests.authentications", "feature"))
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/authentications.py", line 1228, in feature
    Feature(test=verification_cooldown)(rbac=rbac, servers=servers, node=node)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/authentications.py", line 1203, in verification_cooldown
    scenario(server="openldap1", rbac=rbac)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/authentications.py", line 1184, in scenario
    login_and_execute_query(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/authentications.py", line 36, in login_and_execute_query
    self.context.node.query(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py", line 1172, in query
    assert r.exitcode == exitcode, error(r.output)
           ^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert r.exitcode == exitcode, error(r.output)

Description
  1

Assertion values
  assert r.exitcode == exitcode, error(r.output)
         ^ is 
  assert r.exitcode == exitcode, error(r.output)
         ^ is = 0
  assert r.exitcode == exitcode, error(r.output)
                       ^ is 4
  assert r.exitcode == exitcode, error(r.output)
                    ^ is = False
  assert r.exitcode == exitcode, error(r.output)
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py', line 1172 in 'query'

1164\|                      **kwargs,
1165\|                  )
1166\|  
1167\|          if no_checks:
1168\|              return r
1169\|  
1170\|          if exitcode is not None:
1171\|              with Then(f"exitcode should be {exitcode}") if steps else NullStep():
1172\|>                 assert r.exitcode == exitcode, error(r.output)
1173\|  
1174\|          if message is not None:
1175\|              with (
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown reset when invalid password is providedXFail 22s 114ms
https://github.com/ClickHouse/ClickHouse/issues/51323
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 110, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/argparser.py", line 171, in capture_cluster_args
    return func(self, cluster_args=cluster_args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 105, in regression
    Feature(run=load("ldap.authentication.tests.authentications", "feature"))
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/authentications.py", line 1228, in feature
    Feature(test=verification_cooldown)(rbac=rbac, servers=servers, node=node)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/authentications.py", line 1203, in verification_cooldown
    scenario(server="openldap1", rbac=rbac)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/authentications.py", line 1184, in scenario
    login_and_execute_query(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/authentications.py", line 36, in login_and_execute_query
    self.context.node.query(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py", line 1172, in query
    assert r.exitcode == exitcode, error(r.output)
           ^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert r.exitcode == exitcode, error(r.output)

Description
  1

Assertion values
  assert r.exitcode == exitcode, error(r.output)
         ^ is 
  assert r.exitcode == exitcode, error(r.output)
         ^ is = 0
  assert r.exitcode == exitcode, error(r.output)
                       ^ is 4
  assert r.exitcode == exitcode, error(r.output)
                    ^ is = False
  assert r.exitcode == exitcode, error(r.output)
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py', line 1172 in 'query'

1164\|                      **kwargs,
1165\|                  )
1166\|  
1167\|          if no_checks:
1168\|              return r
1169\|  
1170\|          if exitcode is not None:
1171\|              with Then(f"exitcode should be {exitcode}") if steps else NullStep():
1172\|>                 assert r.exitcode == exitcode, error(r.output)
1173\|  
1174\|          if message is not None:
1175\|              with (

Results

Test Name Result Duration
/authentication OK 35m 28s
/authentication/sanity OK 1s 474ms
/authentication/multiple servers OK 21s 882ms
/authentication/connection protocols OK 7m 17s
/authentication/connection protocols/plain text OK 21s 934ms
/authentication/connection protocols/plain text with custom port OK 21s 993ms
/authentication/connection protocols/starttls OK 1m 27s
/authentication/connection protocols/starttls with custom port OK 21s 850ms
/authentication/connection protocols/tls OK 1m 27s
/authentication/connection protocols/tls cipher suite XFail 21s 902ms
/authentication/connection protocols/tls enable tls default yes OK 21s 898ms
/authentication/connection protocols/tls minimum protocol version OK 1m 49s
/authentication/connection protocols/tls require cert default demand XFail 21s 872ms
/authentication/connection protocols/tls with custom port OK 21s 894ms
/authentication/server config OK 4m 21s
/authentication/server config/auth dn value OK 21s 916ms
/authentication/server config/empty ca cert dir OK 21s 975ms
/authentication/server config/empty ca cert file OK 21s 919ms
/authentication/server config/empty host OK 21s 875ms
/authentication/server config/empty server name OK 2s 79ms
/authentication/server config/invalid auth dn prefix OK 21s 954ms
/authentication/server config/invalid auth dn suffix OK 21s 871ms
/authentication/server config/invalid enable tls value OK 21s 956ms
/authentication/server config/invalid host OK 21s 943ms
/authentication/server config/invalid port OK 21s 908ms
/authentication/server config/invalid tls require cert value OK 21s 972ms
/authentication/server config/invalid verification cooldown value OK 12s 476ms
/authentication/server config/missing host OK 21s 978ms
/authentication/server config/syntax OK 5s 882ms
/authentication/user config OK 1m 27s
/authentication/user config/empty server name OK 7s 991ms
/authentication/user config/empty server not defined OK 22s 5ms
/authentication/user config/empty user name OK 5s 961ms
/authentication/user config/ldap and password OK 7s 969ms
/authentication/user config/multiple servers OK 21s 995ms
/authentication/user config/valid user config OK 22s 20ms
/authentication/user authentications OK 18m 31s
/authentication/user authentications/rbac=False OK 11m 48s
/authentication/user authentications/rbac=False/add user to ldap and login OK 17s 177ms
/authentication/user authentications/rbac=False/empty username and empty password OK 77ms
/authentication/user authentications/rbac=False/invalid long username and valid short password OK 16s 101ms
/authentication/user authentications/rbac=False/invalid username and valid password OK 16s 162ms
/authentication/user authentications/rbac=False/login after clickhouse server is restarted OK 29s 862ms
/authentication/user authentications/rbac=False/login after ldap server is restarted OK 19s 272ms
/authentication/user authentications/rbac=False/login after user cn changed in ldap OK 17s 98ms
/authentication/user authentications/rbac=False/login after user is deleted from ldap OK 16s 59ms
/authentication/user authentications/rbac=False/login after user password changed in ldap OK 16s 118ms
/authentication/user authentications/rbac=False/parallel login OK 32s 221ms
/authentication/user authentications/rbac=False/valid ascii username and utf8 password OK 16s 2ms
/authentication/user authentications/rbac=False/valid long username and short password OK 16s 33ms
/authentication/user authentications/rbac=False/valid short username and invalid long password OK 15s 948ms
/authentication/user authentications/rbac=False/valid short username and long password OK 16s 17ms
/authentication/user authentications/rbac=False/valid username and invalid empty password OK 15s 994ms
/authentication/user authentications/rbac=False/valid username and invalid password OK 16s 7ms
/authentication/user authentications/rbac=False/valid username and password OK 16s 66ms
/authentication/user authentications/rbac=False/valid username and password invalid server OK 15s 954ms
/authentication/user authentications/rbac=False/valid username with valid empty password OK 16s 9ms
/authentication/user authentications/rbac=False/valid utf8 username and ascii password OK 16s 65ms
/authentication/user authentications/rbac=False/verification cooldown OK 6m 2s
/authentication/user authentications/rbac=False/verification cooldown/default verification cooldown value OK 22s 66ms
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown reset when invalid password is provided XFail 22s 216ms
/authentication/user authentications/rbac=False/verification cooldown/valid verification cooldown value cn change OK 22s 60ms
/authentication/user authentications/rbac=False/verification cooldown/valid verification cooldown value ldap unavailable OK 27s 417ms
/authentication/user authentications/rbac=False/verification cooldown/valid verification cooldown value password change OK 21s 65ms
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown performance OK 1m 48s
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown reset on server auth dn prefix parameter change OK 34s 824ms
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown reset on server auth dn suffix parameter change OK 34s 869ms
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown reset on server host parameter change OK 34s 977ms
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown reset on server port parameter change OK 34s 884ms
/authentication/user authentications/rbac=True OK 6m 43s
/authentication/user authentications/rbac=True/add user to ldap and login OK 247ms
/authentication/user authentications/rbac=True/empty username and empty password OK 74ms
/authentication/user authentications/rbac=True/invalid long username and valid short password OK 234ms
/authentication/user authentications/rbac=True/invalid username and valid password OK 230ms
/authentication/user authentications/rbac=True/login after clickhouse server is restarted OK 13s 997ms
/authentication/user authentications/rbac=True/login after ldap server is restarted OK 3s 467ms
/authentication/user authentications/rbac=True/login after user cn changed in ldap OK 334ms
/authentication/user authentications/rbac=True/login after user is deleted from ldap OK 316ms
/authentication/user authentications/rbac=True/login after user password changed in ldap OK 390ms
/authentication/user authentications/rbac=True/parallel login OK 17s 501ms
/authentication/user authentications/rbac=True/valid ascii username and utf8 password OK 251ms
/authentication/user authentications/rbac=True/valid long username and short password OK 248ms
/authentication/user authentications/rbac=True/valid short username and invalid long password OK 219ms
/authentication/user authentications/rbac=True/valid short username and long password OK 244ms
/authentication/user authentications/rbac=True/valid username and invalid empty password OK 223ms
/authentication/user authentications/rbac=True/valid username and invalid password OK 225ms
/authentication/user authentications/rbac=True/valid username and password OK 238ms
/authentication/user authentications/rbac=True/valid username and password invalid server OK 186ms
/authentication/user authentications/rbac=True/valid username with valid empty password OK 226ms
/authentication/user authentications/rbac=True/valid utf8 username and ascii password OK 245ms
/authentication/user authentications/rbac=True/verification cooldown OK 5m 59s
/authentication/user authentications/rbac=True/verification cooldown/default verification cooldown value OK 21s 956ms
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown reset when invalid password is provided XFail 22s 114ms
/authentication/user authentications/rbac=True/verification cooldown/valid verification cooldown value cn change OK 22s 16ms
/authentication/user authentications/rbac=True/verification cooldown/valid verification cooldown value ldap unavailable OK 27s 308ms
/authentication/user authentications/rbac=True/verification cooldown/valid verification cooldown value password change OK 20s 982ms
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown performance OK 1m 46s
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown reset on server auth dn prefix parameter change OK 34s 663ms
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown reset on server auth dn suffix parameter change OK 34s 744ms
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown reset on server host parameter change OK 34s 693ms
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown reset on server port parameter change OK 34s 779ms
/authentication/multiple authentications methods OK 6s 402ms
/authentication/multiple authentications methods/ldap with other auth methods OK 1s 560ms

Generated by TestFlows Open-Source Test Framework v2.0.250110.1002922