Tiered Storage Test Run Report

DateJun 27, 2025 13:09
Duration1h 28m
Framework TestFlows 2.0.250110.1002922

Artifacts

Test artifacts can be found at https://altinity-build-artifacts.s3.amazonaws.com/index.html#0/d1f73c287081665b9e9ca2767a633a3cc8a41f08/regression/x86_64/with_analyzer/zookeeper/without_thread_fuzzer/tiered_storage/minio/

Attributes

projectAltinity/ClickHouse
project.id159717931
packagehttps://s3.amazonaws.com/altinity-build-artifacts/24.8/d1f73c287081665b9e9ca2767a633a3cc8a41f08/package_release/clickhouse-common-static_24.8.14.10501.altinitytest_amd64.deb
version24.8.14.10501.altinitytest
user.nameEnmk
repositoryhttps://github.com/Altinity/clickhouse-regression
commit.hashe3c00be97a045aa04e9d1a6ec50cc64f4c387b70
job.nameTieredStorage
job.retry1
job.urlhttps://github.com/Altinity/ClickHouse/actions/runs/15925737638
archx86_64
localTrue
clickhouse_versionNone
clickhouse_pathhttps://s3.amazonaws.com/altinity-build-artifacts/24.8/d1f73c287081665b9e9ca2767a633a3cc8a41f08/package_release/clickhouse-common-static_24.8.14.10501.altinitytest_amd64.deb
as_binaryFalse
base_osNone
keeper_pathNone
zookeeper_versionNone
use_keeperFalse
stressFalse
collect_service_logsTrue
thread_fuzzerFalse
with_analyzerTrue
reuse_envFalse
with_minioTrue
with_s3amazonFalse
with_s3gcsFalse
aws_s3_access_keySecret(name='aws_s3_access_key')
aws_s3_key_idSecret(name='aws_s3_key_id')
aws_s3_uriSecret(name='aws_s3_uri')
gcs_uriSecret(name='gcs_uri')
gcs_key_idSecret(name='gcs_key_id')
gcs_key_secretSecret(name='gcs_key_secret')

Summary

97.4%OK
2.6%Known

Statistics

Units OK Fail XFail
Modules
1
1
Features
1
1
Scenarios
94
90
4
Checks
92
92
Examples
84
81
3
Steps
12663
12444
219

Known Fails

Test NameResultMessage
/tiered storage/with minio/attach or replace partition different policiesXFail 444ms
known issue
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/tiered_storage/regression.py", line 436, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/tiered_storage/../helpers/argparser.py", line 171, in capture_cluster_args
    return func(self, cluster_args=cluster_args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/tiered_storage/regression.py", line 425, in regression
    Feature(name, test=feature)(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/tiered_storage/regression.py", line 224, in feature
    Scenario(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/tiered_storage/../tiered_storage/tests/attach_or_replace_partition_different_policies.py", line 54, in scenario
    node.query(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/tiered_storage/../helpers/cluster.py", line 1180, in query
    assert message in r.output, error(r.output)
           ^^^^^^^^^^^^^^^^^^^
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert message in r.output, error(r.output)

Assertion values
  assert message in r.output, error(r.output)
         ^ is 'Exception:'
  assert message in r.output, error(r.output)
                    ^ is 
  assert message in r.output, error(r.output)
                    ^ is = ''
  assert message in r.output, error(r.output)
                 ^ is = False
  assert message in r.output, error(r.output)
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/tiered_storage/../helpers/cluster.py', line 1180 in 'query'

1172\|                  assert r.exitcode == exitcode, error(r.output)
1173\|  
1174\|          if message is not None:
1175\|              with (
1176\|                  Then(f"output should contain message", description=message)
1177\|                  if steps
1178\|                  else NullStep()
1179\|              ):
1180\|>                 assert message in r.output, error(r.output)
1181\|  
1182\|          if not ignore_exception:
1183\|              if message is None or "Exception:" not in message:
/tiered storage/with minio/double move while selectXFail 2s 407ms
not yet supported
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/tiered_storage/regression.py", line 436, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/tiered_storage/../helpers/argparser.py", line 171, in capture_cluster_args
    return func(self, cluster_args=cluster_args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/tiered_storage/regression.py", line 425, in regression
    Feature(name, test=feature)(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/tiered_storage/regression.py", line 295, in feature
    Scenario(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/tiered_storage/../tiered_storage/tests/double_move_while_select.py", line 120, in scenario
    assert r == [
           ^^^^^^
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert r == [
      "jbod1" if positive else "external"
  ], error()

Assertion values
  assert r == [
         ^ is ['jbod1']
      "jbod1" if positive else "external"
  ], error()
  assert r == [
      "jbod1" if positive else "external"
                 ^ is 0
  ], error()
  assert r == [
      "jbod1" if positive else "external"
      ^ is = 'external'
  ], error()
  assert r == [
              ^ is = ['external']
      "jbod1" if positive else "external"
  ], error()
  assert r == [
           ^ is = False
    @@ -1 +1 @@
    -['jbod1']
    +['external']
      "jbod1" if positive else "external"
  ], error()
  assert r == [
  ^ is False
      "jbod1" if positive else "external"
  ], error()

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/tiered_storage/../tiered_storage/tests/double_move_while_select.py', line 122 in 'scenario'

114\|                                      .output.strip()
115\|                                      .splitlines()
116\|                                  )
117\|                                  with Then(
118\|                                      "if select locked old part on the destination disk the move shall fail"
119\|                                  ):
120\|                                      assert r == [
121\|                                          "jbod1" if positive else "external"
122\|>                                     ], error()
123\|                          finally:
124\|                              with Finally("I join the thread"):
125\|                                  task.result(timeout=60)
/tiered storage/with minio/manual move with downtimeXFail 903ms
https://altinity.atlassian.net/browse/CH-124
AssertionError
Traceback (most recent call last):
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/tiered_storage/regression.py", line 436, in 
    regression()
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/tiered_storage/../helpers/argparser.py", line 171, in capture_cluster_args
    return func(self, cluster_args=cluster_args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/tiered_storage/regression.py", line 425, in regression
    Feature(name, test=feature)(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/tiered_storage/regression.py", line 307, in feature
    Scenario(
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/tiered_storage/../tiered_storage/tests/manual_move_with_downtime.py", line 83, in scenario
    node.query(f"ALTER TABLE {name} ATTACH PARTITION 201904")
  File "/home/ubuntu/_work/ClickHouse/ClickHouse/tiered_storage/../helpers/cluster.py", line 1188, in query
    assert False, error(r.output)
           ^^^^^
AssertionError: Oops! Assertion failed

The following assertion was not satisfied
  assert False, error(r.output)

Description
  Received exception from server (version 24.8.14):
Code: 27. DB::Exception: Received from localhost:9000. DB::Exception: Cannot parse input: expected '\n' before: 'checksums format version: 4\nT\xf3\xb8\xdc#a\x90tC߉^e\|\xaf\x82\xed\0\0\0\xe4\0\0\0\xf1;\tcount.txtl\x9cU\xbd隓\|\xa9STX\xf4\xff\0\bdata.bin>\xcfM\xd8\xfc\xd5,\xcd\xcc\xed\xbb\x85ϱ>\n\x9e\xa1\xe3\x8e3\x91\xc6vYÿ8\xaal\xfc\n,\0\xf1jcmrk3:}]\xbdX\xf1\xd3u\x86r\xc0)\f%\x93\xc2P_\x8cN'. (CANNOT_PARSE_INPUT_ASSERTION_FAILED)
(query: ALTER TABLE manual_move_mt ATTACH PARTITION 201904
)

Assertion values
  assert False, error(r.output)
  ^ is False

Where
  File '/home/ubuntu/_work/ClickHouse/ClickHouse/tiered_storage/../helpers/cluster.py', line 1188 in 'query'

1180\|                  assert message in r.output, error(r.output)
1181\|  
1182\|          if not ignore_exception:
1183\|              if message is None or "Exception:" not in message:
1184\|                  with Then("check if output has exception") if steps else NullStep():
1185\|                      if "Exception:" in r.output:
1186\|                          if raise_on_exception:
1187\|                              raise QueryRuntimeException(r.output)
1188\|>                         assert False, error(r.output)
1189\|  
1190\|          return r
1191\|

Results

Test Name Result Duration
/tiered storage OK 1h 28m
/tiered storage/with minio OK 1h 27m
/tiered storage/with minio/startup and queries OK 26s 191ms
/tiered storage/with minio/metadata OK 3s 234ms
/tiered storage/with minio/no changes to queries OK 3s 850ms
/tiered storage/with minio/volume configuration either max_data_part_size_bytes or max_data_part_size_ratio OK 15s 6ms
/tiered storage/with minio/system tables OK 175ms
/tiered storage/with minio/system․detached_parts OK 1s 630ms
/tiered storage/with minio/attach or replace partition different policies XFail 444ms
/tiered storage/with minio/query parser OK 642ms
/tiered storage/with minio/keep free space OK 10s 750ms
/tiered storage/with minio/no warning about zero max data part size OK 513ms
/tiered storage/with minio/round robin OK 1s 236ms
/tiered storage/with minio/max data part size OK 3s 223ms
/tiered storage/with minio/jbod overflow OK 30s 596ms
/tiered storage/with minio/start stop moves OK 9s 784ms
/tiered storage/with minio/mutate to another disk OK 7s 258ms
/tiered storage/with minio/alter table policy OK 15s 164ms
/tiered storage/with minio/simple replication and moves OK 39s 169ms
/tiered storage/with minio/download appropriate disk OK 3s 603ms
/tiered storage/with minio/download appropriate disk advanced OK 17s 809ms
/tiered storage/with minio/download appropriate disk max data part size OK 9s 831ms
/tiered storage/with minio/rename OK 4s 339ms
/tiered storage/with minio/freeze OK 3s 639ms
/tiered storage/with minio/double move while select XFail 2s 407ms
/tiered storage/with minio/background move OK 3m 50s
/tiered storage/with minio/alter move OK 9m 10s
/tiered storage/with minio/manual move with downtime XFail 903ms
/tiered storage/with minio/merge parts different volumes OK 11s 542ms
/tiered storage/with minio/merge parts different volumes no space OK 5s 113ms
/tiered storage/with minio/ttl moves OK 1h 4m
/tiered storage/with minio/change config norestart OK 5m 33s

Generated by TestFlows Open-Source Test Framework v2.0.250110.1002922