Copyright 2025, Altinity Inc.. All Rights Reserved. All information contained herein is, and remains the property of Altinity Inc.. Any dissemination of this information or reproduction of this material is strictly forbidden unless prior written permission is obtained from Altinity Inc..

Authentication Test Run Report

DateApr 04, 2025 16:50
Duration39m 43s
Framework TestFlows 2.0.250110.1002922

Artifacts

Test artifacts can be found at https://altinity-build-artifacts.s3.amazonaws.com/index.html#697/be49e038b8fbf83bb8a525931ba639d034980d8d/regression/aarch64/with_analyzer/zookeeper/without_thread_fuzzer/ldap/authentication/

Attributes

projectAltinity/ClickHouse
project.id159717931
packagehttps://s3.amazonaws.com/altinity-build-artifacts/PRs/697/be49e038b8fbf83bb8a525931ba639d034980d8d/package_aarch64/clickhouse-common-static_24.12.2.20271.altinityantalya_arm64.deb
version24.12.2.20271.altinityantalya
user.namearthurpassos
repositoryhttps://github.com/Altinity/clickhouse-regression
commit.hashbd31e738c0cedaca253d15a05ed245c41b6e0b6a
job.nameLDAP
job.retry1
job.urlhttps://github.com/Altinity/ClickHouse/actions/runs/14269104049
archaarch64
localTrue
clickhouse_versionNone
clickhouse_pathhttps://s3.amazonaws.com/altinity-build-artifacts/PRs/697/be49e038b8fbf83bb8a525931ba639d034980d8d/package_aarch64/clickhouse-common-static_24.12.2.20271.altinityantalya_arm64.deb
as_binaryFalse
base_osNone
keeper_pathNone
zookeeper_versionNone
use_keeperFalse
stressFalse
collect_service_logsTrue
thread_fuzzerFalse
with_analyzerTrue
reuse_envFalse

Summary

93.7%OK
6.3%Known

Statistics

Units OK Fail Error XFail
Features
8
8
Scenarios
93
89
4
Examples
25
21
4
Steps
6327
6305
16
6

Known Fails

Test NameResultMessage
/authentication/connection protocols/tls cipher suiteXFail 24s 409ms
can't get it to work
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 110, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/argparser.py", line 171, in capture_cluster_args
    return func(self, cluster_args=cluster_args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 102, in regression
    Feature(run=load("ldap.authentication.tests.connections", "feature"))
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/connections.py", line 311, in feature
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/connections.py", line 242, in tls_cipher_suite
    login(servers, *users)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/common.py", line 438, in login
    current().context.node.query(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py", line 1188, in query
    assert False, error(r.output)
           ^^^^^
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert False, error(r.output)

Description
  Code: 516. DB::Exception: Received from localhost:9000. DB::Exception: user4: Authentication failed: password is incorrect, or there is no user with such name.. (AUTHENTICATION_FAILED)

Assertion values
  assert False, error(r.output)
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py', line 1188 in 'query'

1180\|                  assert message in r.output, error(r.output)
1181\|  
1182\|          if not ignore_exception:
1183\|              if message is None or "Exception:" not in message:
1184\|                  with Then("check if output has exception") if steps else NullStep():
1185\|                      if "Exception:" in r.output:
1186\|                          if raise_on_exception:
1187\|                              raise QueryRuntimeException(r.output)
1188\|>                         assert False, error(r.output)
1189\|  
1190\|          return r
1191\|
/authentication/connection protocols/tls require cert default demandXFail 24s 709ms
can't be tested with self-signed certificates
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 110, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/argparser.py", line 171, in capture_cluster_args
    return func(self, cluster_args=cluster_args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 102, in regression
    Feature(run=load("ldap.authentication.tests.connections", "feature"))
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/connections.py", line 311, in feature
    scenario()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/connections.py", line 199, in tls_require_cert_default_demand
    login(servers, *users)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/common.py", line 438, in login
    current().context.node.query(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py", line 1188, in query
    assert False, error(r.output)
           ^^^^^
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert False, error(r.output)

Description
  Code: 516. DB::Exception: Received from localhost:9000. DB::Exception: user2: Authentication failed: password is incorrect, or there is no user with such name.. (AUTHENTICATION_FAILED)

Assertion values
  assert False, error(r.output)
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py', line 1188 in 'query'

1180\|                  assert message in r.output, error(r.output)
1181\|  
1182\|          if not ignore_exception:
1183\|              if message is None or "Exception:" not in message:
1184\|                  with Then("check if output has exception") if steps else NullStep():
1185\|                      if "Exception:" in r.output:
1186\|                          if raise_on_exception:
1187\|                              raise QueryRuntimeException(r.output)
1188\|>                         assert False, error(r.output)
1189\|  
1190\|          return r
1191\|
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown reset when invalid password is providedXFail 25s 892ms
https://github.com/ClickHouse/ClickHouse/issues/51323
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 110, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/argparser.py", line 171, in capture_cluster_args
    return func(self, cluster_args=cluster_args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 105, in regression
    Feature(run=load("ldap.authentication.tests.authentications", "feature"))
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/authentications.py", line 1228, in feature
    Feature(test=verification_cooldown)(rbac=rbac, servers=servers, node=node)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/authentications.py", line 1203, in verification_cooldown
    scenario(server="openldap1", rbac=rbac)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/authentications.py", line 1184, in scenario
    login_and_execute_query(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/authentications.py", line 36, in login_and_execute_query
    self.context.node.query(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py", line 1172, in query
    assert r.exitcode == exitcode, error(r.output)
           ^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert r.exitcode == exitcode, error(r.output)

Description
  1

Assertion values
  assert r.exitcode == exitcode, error(r.output)
         ^ is 
  assert r.exitcode == exitcode, error(r.output)
         ^ is = 0
  assert r.exitcode == exitcode, error(r.output)
                       ^ is 4
  assert r.exitcode == exitcode, error(r.output)
                    ^ is = False
  assert r.exitcode == exitcode, error(r.output)
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py', line 1172 in 'query'

1164\|                      **kwargs,
1165\|                  )
1166\|  
1167\|          if no_checks:
1168\|              return r
1169\|  
1170\|          if exitcode is not None:
1171\|              with Then(f"exitcode should be {exitcode}") if steps else NullStep():
1172\|>                 assert r.exitcode == exitcode, error(r.output)
1173\|  
1174\|          if message is not None:
1175\|              with (
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown reset when invalid password is providedXFail 25s 833ms
https://github.com/ClickHouse/ClickHouse/issues/51323
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 110, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/argparser.py", line 171, in capture_cluster_args
    return func(self, cluster_args=cluster_args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/regression.py", line 105, in regression
    Feature(run=load("ldap.authentication.tests.authentications", "feature"))
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/authentications.py", line 1228, in feature
    Feature(test=verification_cooldown)(rbac=rbac, servers=servers, node=node)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/authentications.py", line 1203, in verification_cooldown
    scenario(server="openldap1", rbac=rbac)
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/authentications.py", line 1184, in scenario
    login_and_execute_query(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../ldap/authentication/tests/authentications.py", line 36, in login_and_execute_query
    self.context.node.query(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py", line 1172, in query
    assert r.exitcode == exitcode, error(r.output)
           ^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert r.exitcode == exitcode, error(r.output)

Description
  1

Assertion values
  assert r.exitcode == exitcode, error(r.output)
         ^ is 
  assert r.exitcode == exitcode, error(r.output)
         ^ is = 0
  assert r.exitcode == exitcode, error(r.output)
                       ^ is 4
  assert r.exitcode == exitcode, error(r.output)
                    ^ is = False
  assert r.exitcode == exitcode, error(r.output)
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/ldap/authentication/../../helpers/cluster.py', line 1172 in 'query'

1164\|                      **kwargs,
1165\|                  )
1166\|  
1167\|          if no_checks:
1168\|              return r
1169\|  
1170\|          if exitcode is not None:
1171\|              with Then(f"exitcode should be {exitcode}") if steps else NullStep():
1172\|>                 assert r.exitcode == exitcode, error(r.output)
1173\|  
1174\|          if message is not None:
1175\|              with (

Results

Test Name Result Duration
/authentication OK 39m 43s
/authentication/sanity OK 1s 767ms
/authentication/multiple servers OK 24s 450ms
/authentication/connection protocols OK 8m 7s
/authentication/connection protocols/plain text OK 23s 348ms
/authentication/connection protocols/plain text with custom port OK 24s 493ms
/authentication/connection protocols/starttls OK 1m 39s
/authentication/connection protocols/starttls with custom port OK 24s 742ms
/authentication/connection protocols/tls OK 1m 37s
/authentication/connection protocols/tls cipher suite XFail 24s 409ms
/authentication/connection protocols/tls enable tls default yes OK 24s 561ms
/authentication/connection protocols/tls minimum protocol version OK 1m 58s
/authentication/connection protocols/tls require cert default demand XFail 24s 709ms
/authentication/connection protocols/tls with custom port OK 24s 624ms
/authentication/server config OK 4m 53s
/authentication/server config/auth dn value OK 24s 937ms
/authentication/server config/empty ca cert dir OK 24s 483ms
/authentication/server config/empty ca cert file OK 24s 779ms
/authentication/server config/empty host OK 24s 560ms
/authentication/server config/empty server name OK 2s 114ms
/authentication/server config/invalid auth dn prefix OK 24s 614ms
/authentication/server config/invalid auth dn suffix OK 24s 832ms
/authentication/server config/invalid enable tls value OK 24s 881ms
/authentication/server config/invalid host OK 24s 628ms
/authentication/server config/invalid port OK 24s 640ms
/authentication/server config/invalid tls require cert value OK 24s 732ms
/authentication/server config/invalid verification cooldown value OK 12s 721ms
/authentication/server config/missing host OK 23s 877ms
/authentication/server config/syntax OK 7s 673ms
/authentication/user config OK 1m 40s
/authentication/user config/empty server name OK 8s 530ms
/authentication/user config/empty server not defined OK 24s 645ms
/authentication/user config/empty user name OK 9s 632ms
/authentication/user config/ldap and password OK 6s 570ms
/authentication/user config/multiple servers OK 24s 846ms
/authentication/user config/valid user config OK 26s 80ms
/authentication/user authentications OK 20m 51s
/authentication/user authentications/rbac=False OK 13m 14s
/authentication/user authentications/rbac=False/add user to ldap and login OK 18s 292ms
/authentication/user authentications/rbac=False/empty username and empty password OK 110ms
/authentication/user authentications/rbac=False/invalid long username and valid short password OK 17s 365ms
/authentication/user authentications/rbac=False/invalid username and valid password OK 17s 482ms
/authentication/user authentications/rbac=False/login after clickhouse server is restarted OK 31s 847ms
/authentication/user authentications/rbac=False/login after ldap server is restarted OK 21s 915ms
/authentication/user authentications/rbac=False/login after user cn changed in ldap OK 17s 995ms
/authentication/user authentications/rbac=False/login after user is deleted from ldap OK 18s 51ms
/authentication/user authentications/rbac=False/login after user password changed in ldap OK 18s 204ms
/authentication/user authentications/rbac=False/parallel login OK 38s 71ms
/authentication/user authentications/rbac=False/valid ascii username and utf8 password OK 17s 961ms
/authentication/user authentications/rbac=False/valid long username and short password OK 18s 158ms
/authentication/user authentications/rbac=False/valid short username and invalid long password OK 18s 77ms
/authentication/user authentications/rbac=False/valid short username and long password OK 17s 19ms
/authentication/user authentications/rbac=False/valid username and invalid empty password OK 16s 905ms
/authentication/user authentications/rbac=False/valid username and invalid password OK 17s 15ms
/authentication/user authentications/rbac=False/valid username and password OK 17s 54ms
/authentication/user authentications/rbac=False/valid username and password invalid server OK 17s 130ms
/authentication/user authentications/rbac=False/valid username with valid empty password OK 17s 58ms
/authentication/user authentications/rbac=False/valid utf8 username and ascii password OK 17s 203ms
/authentication/user authentications/rbac=False/verification cooldown OK 6m 55s
/authentication/user authentications/rbac=False/verification cooldown/default verification cooldown value OK 24s 819ms
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown reset when invalid password is provided XFail 25s 892ms
/authentication/user authentications/rbac=False/verification cooldown/valid verification cooldown value cn change OK 24s 761ms
/authentication/user authentications/rbac=False/verification cooldown/valid verification cooldown value ldap unavailable OK 30s 680ms
/authentication/user authentications/rbac=False/verification cooldown/valid verification cooldown value password change OK 24s 837ms
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown performance OK 2m 11s
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown reset on server auth dn prefix parameter change OK 38s 634ms
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown reset on server auth dn suffix parameter change OK 39s 578ms
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown reset on server host parameter change OK 37s 515ms
/authentication/user authentications/rbac=False/verification cooldown/verification cooldown reset on server port parameter change OK 37s 659ms
/authentication/user authentications/rbac=True OK 7m 36s
/authentication/user authentications/rbac=True/add user to ldap and login OK 354ms
/authentication/user authentications/rbac=True/empty username and empty password OK 100ms
/authentication/user authentications/rbac=True/invalid long username and valid short password OK 308ms
/authentication/user authentications/rbac=True/invalid username and valid password OK 338ms
/authentication/user authentications/rbac=True/login after clickhouse server is restarted OK 15s 138ms
/authentication/user authentications/rbac=True/login after ldap server is restarted OK 4s 214ms
/authentication/user authentications/rbac=True/login after user cn changed in ldap OK 372ms
/authentication/user authentications/rbac=True/login after user is deleted from ldap OK 371ms
/authentication/user authentications/rbac=True/login after user password changed in ldap OK 505ms
/authentication/user authentications/rbac=True/parallel login OK 21s 107ms
/authentication/user authentications/rbac=True/valid ascii username and utf8 password OK 308ms
/authentication/user authentications/rbac=True/valid long username and short password OK 304ms
/authentication/user authentications/rbac=True/valid short username and invalid long password OK 255ms
/authentication/user authentications/rbac=True/valid short username and long password OK 294ms
/authentication/user authentications/rbac=True/valid username and invalid empty password OK 262ms
/authentication/user authentications/rbac=True/valid username and invalid password OK 263ms
/authentication/user authentications/rbac=True/valid username and password OK 346ms
/authentication/user authentications/rbac=True/valid username and password invalid server OK 195ms
/authentication/user authentications/rbac=True/valid username with valid empty password OK 284ms
/authentication/user authentications/rbac=True/valid utf8 username and ascii password OK 326ms
/authentication/user authentications/rbac=True/verification cooldown OK 6m 45s
/authentication/user authentications/rbac=True/verification cooldown/default verification cooldown value OK 24s 676ms
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown reset when invalid password is provided XFail 25s 833ms
/authentication/user authentications/rbac=True/verification cooldown/valid verification cooldown value cn change OK 25s 799ms
/authentication/user authentications/rbac=True/verification cooldown/valid verification cooldown value ldap unavailable OK 30s 67ms
/authentication/user authentications/rbac=True/verification cooldown/valid verification cooldown value password change OK 25s 724ms
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown performance OK 2m 2s
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown reset on server auth dn prefix parameter change OK 37s 988ms
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown reset on server auth dn suffix parameter change OK 38s 147ms
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown reset on server host parameter change OK 37s 21ms
/authentication/user authentications/rbac=True/verification cooldown/verification cooldown reset on server port parameter change OK 37s 396ms
/authentication/multiple authentications methods OK 8s 249ms
/authentication/multiple authentications methods/ldap with other auth methods OK 1s 837ms

Generated by TestFlows Open-Source Test Framework v2.0.250110.1002922