このページの翻訳:
- 日本語 (ja)
- English (en)
最近の更新
Tag Cloud
このページへのアクセス
今日: 4 / 昨日: 19
総計: 982
- Dokuwiki.fl8.jp(59)
- 75 Proftpd(21)
- DokuWiki(11)
- 06 Dokuwiki 初期設定(10)
最近の更新
このページへのアクセス
今日: 4 / 昨日: 19
総計: 982
# ceph-volume lvm prepare --bluestore --data ssd/data --block.wal optane/waldb --block.db optane/rocksdb --> RuntimeError: skipping ssd/data, it is already prepared [root@cephdev001 my-cluster]# lvremove -f /dev/ssd/data Logical volume "data" successfully removed [root@cephdev001 my-cluster]# lvcreate -n data -l 100%Free ssd Logical volume "data" created. [root@cephdev001 my-cluster]# ceph-volume lvm prepare --bluestore --data ssd/data --block.wal optane/waldb --block.db optane/rocksdb Running command: /usr/bin/ceph-authtool --gen-print-key Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring -i - osd new 2dd353ac-dcb8-48a8-88cc-cdf96b6ecfcf Running command: /usr/bin/ceph-authtool --gen-print-key Running command: /usr/bin/mount -t tmpfs tmpfs /var/lib/ceph/osd/ceph-0 Running command: /usr/bin/chown -h ceph:ceph /dev/ssd/data Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 Running command: /usr/bin/ln -s /dev/ssd/data /var/lib/ceph/osd/ceph-0/block Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring mon getmap -o /var/lib/ceph/osd/ceph-0/activate.monmap stderr: 2020-09-02 18:30:09.744 7f8fd77c0700 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,: (2) No such file or directory 2020-09-02 18:30:09.744 7f8fd77c0700 -1 AuthRegistry(0x7f8fd00656b8) no keyring found at /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,, disabling cephx stderr: got monmap epoch 1 Running command: /usr/bin/ceph-authtool /var/lib/ceph/osd/ceph-0/keyring --create-keyring --name osd.0 --add-key AQAgZk9fqxUgKRAA8gJEbwtnuVb91YJHVZgm/Q== stdout: creating /var/lib/ceph/osd/ceph-0/keyring added entity osd.0 auth(key=AQAgZk9fqxUgKRAA8gJEbwtnuVb91YJHVZgm/Q==) Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0/keyring Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0/ Running command: /usr/bin/chown -h ceph:ceph /dev/optane/waldb Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 Running command: /usr/bin/chown -h ceph:ceph /dev/optane/rocksdb Running command: /usr/bin/chown -R ceph:ceph /dev/dm-5 Running command: /usr/bin/ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 0 --monmap /var/lib/ceph/osd/ceph-0/activate.monmap --keyfile - --bluestore-block-wal-path /dev/optane/waldb --bluestore-block-db-path /dev/optane/rocksdb --osd-data /var/lib/ceph/osd/ceph-0/ --osd-uuid 2dd353ac-dcb8-48a8-88cc-cdf96b6ecfcf --setuser ceph --setgroup ceph stderr: 2020-09-02 18:30:10.301 7f600913da80 -1 bluestore(/var/lib/ceph/osd/ceph-0/) _read_fsid unparsable uuid stderr: 2020-09-02 18:30:10.303 7f600913da80 -1 rocksdb: Invalid argument: Can't parse option compaction_threads stderr: 2020-09-02 18:30:10.558 7f600913da80 -1 bluestore(/var/lib/ceph/osd/ceph-0/) mkfs failed, (22) Invalid argument stderr: 2020-09-02 18:30:10.558 7f600913da80 -1 OSD::mkfs: ObjectStore::mkfs failed with error (22) Invalid argument stderr: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/gigantic/release/14.2.11/rpm/el7/BUILD/ceph-14.2.11/src/os/bluestore/BlueStore.cc: In function 'virtual BlueStore::~BlueStore()' thread 7f600913da80 time 2020-09-02 18:30:10.559517 stderr: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/gigantic/release/14.2.11/rpm/el7/BUILD/ceph-14.2.11/src/os/bluestore/BlueStore.cc: 3944: FAILED ceph_assert(bluefs == __null) stderr: ceph version 14.2.11 (f7fdb2f52131f54b891a2ec99d8205561242cdaf) nautilus (stable) stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x14a) [0x56366177c0e5] stderr: 2: (()+0x4d72ad) [0x56366177c2ad] stderr: 3: (BlueStore::~BlueStore()+0x7dd) [0x563661cf15fd] stderr: 4: (BlueStore::~BlueStore()+0x9) [0x563661cf1639] stderr: 5: (OSD::mkfs(CephContext*, ObjectStore*, uuid_d, int)+0x898) [0x563661839d28] stderr: 6: (main()+0x1501) [0x5636617e3481] stderr: 7: (__libc_start_main()+0xf5) [0x7f6004f69555] stderr: 8: (()+0x573c85) [0x563661818c85] stderr: *** Caught signal (Aborted) ** stderr: in thread 7f600913da80 thread_name:ceph-osd stderr: 2020-09-02 18:30:10.567 7f600913da80 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/gigantic/release/14.2.11/rpm/el7/BUILD/ceph-14.2.11/src/os/bluestore/BlueStore.cc: In function 'virtual BlueStore::~BlueStore()' thread 7f600913da80 time 2020-09-02 18:30:10.559517 stderr: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/gigantic/release/14.2.11/rpm/el7/BUILD/ceph-14.2.11/src/os/bluestore/BlueStore.cc: 3944: FAILED ceph_assert(bluefs == __null) stderr: ceph version 14.2.11 (f7fdb2f52131f54b891a2ec99d8205561242cdaf) nautilus (stable) stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x14a) [0x56366177c0e5] stderr: 2: (()+0x4d72ad) [0x56366177c2ad] stderr: 3: (BlueStore::~BlueStore()+0x7dd) [0x563661cf15fd] stderr: 4: (BlueStore::~BlueStore()+0x9) [0x563661cf1639] stderr: 5: (OSD::mkfs(CephContext*, ObjectStore*, uuid_d, int)+0x898) [0x563661839d28] stderr: 6: (main()+0x1501) [0x5636617e3481] stderr: 7: (__libc_start_main()+0xf5) [0x7f6004f69555] stderr: 8: (()+0x573c85) [0x563661818c85] stderr: ceph version 14.2.11 (f7fdb2f52131f54b891a2ec99d8205561242cdaf) nautilus (stable) stderr: 1: (()+0xf630) [0x7f6006189630] stderr: 2: (gsignal()+0x37) [0x7f6004f7d387] stderr: 3: (abort()+0x148) [0x7f6004f7ea78] stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x199) [0x56366177c134] stderr: 5: (()+0x4d72ad) [0x56366177c2ad] stderr: 6: (BlueStore::~BlueStore()+0x7dd) [0x563661cf15fd] stderr: 7: (BlueStore::~BlueStore()+0x9) [0x563661cf1639] stderr: 8: (OSD::mkfs(CephContext*, ObjectStore*, uuid_d, int)+0x898) [0x563661839d28] stderr: 9: (main()+0x1501) [0x5636617e3481] stderr: 10: (__libc_start_main()+0xf5) [0x7f6004f69555] stderr: 11: (()+0x573c85) [0x563661818c85] stderr: 2020-09-02 18:30:10.574 7f600913da80 -1 *** Caught signal (Aborted) ** stderr: in thread 7f600913da80 thread_name:ceph-osd stderr: ceph version 14.2.11 (f7fdb2f52131f54b891a2ec99d8205561242cdaf) nautilus (stable) stderr: 1: (()+0xf630) [0x7f6006189630] stderr: 2: (gsignal()+0x37) [0x7f6004f7d387] stderr: 3: (abort()+0x148) [0x7f6004f7ea78] stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x199) [0x56366177c134] stderr: 5: (()+0x4d72ad) [0x56366177c2ad] stderr: 6: (BlueStore::~BlueStore()+0x7dd) [0x563661cf15fd] stderr: 7: (BlueStore::~BlueStore()+0x9) [0x563661cf1639] stderr: 8: (OSD::mkfs(CephContext*, ObjectStore*, uuid_d, int)+0x898) [0x563661839d28] stderr: 9: (main()+0x1501) [0x5636617e3481] stderr: 10: (__libc_start_main()+0xf5) [0x7f6004f69555] stderr: 11: (()+0x573c85) [0x563661818c85] stderr: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. stderr: -34> 2020-09-02 18:30:10.301 7f600913da80 -1 bluestore(/var/lib/ceph/osd/ceph-0/) _read_fsid unparsable uuid stderr: -5> 2020-09-02 18:30:10.303 7f600913da80 -1 rocksdb: Invalid argument: Can't parse option compaction_threads stderr: -3> 2020-09-02 18:30:10.558 7f600913da80 -1 bluestore(/var/lib/ceph/osd/ceph-0/) mkfs failed, (22) Invalid argument stderr: -2> 2020-09-02 18:30:10.558 7f600913da80 -1 OSD::mkfs: ObjectStore::mkfs failed with error (22) Invalid argument stderr: -1> 2020-09-02 18:30:10.567 7f600913da80 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/gigantic/release/14.2.11/rpm/el7/BUILD/ceph-14.2.11/src/os/bluestore/BlueStore.cc: In function 'virtual BlueStore::~BlueStore()' thread 7f600913da80 time 2020-09-02 18:30:10.559517 stderr: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/gigantic/release/14.2.11/rpm/el7/BUILD/ceph-14.2.11/src/os/bluestore/BlueStore.cc: 3944: FAILED ceph_assert(bluefs == __null) stderr: ceph version 14.2.11 (f7fdb2f52131f54b891a2ec99d8205561242cdaf) nautilus (stable) stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x14a) [0x56366177c0e5] stderr: 2: (()+0x4d72ad) [0x56366177c2ad] stderr: 3: (BlueStore::~BlueStore()+0x7dd) [0x563661cf15fd] stderr: 4: (BlueStore::~BlueStore()+0x9) [0x563661cf1639] stderr: 5: (OSD::mkfs(CephContext*, ObjectStore*, uuid_d, int)+0x898) [0x563661839d28] stderr: 6: (main()+0x1501) [0x5636617e3481] stderr: 7: (__libc_start_main()+0xf5) [0x7f6004f69555] stderr: 8: (()+0x573c85) [0x563661818c85] stderr: 0> 2020-09-02 18:30:10.574 7f600913da80 -1 *** Caught signal (Aborted) ** stderr: in thread 7f600913da80 thread_name:ceph-osd stderr: ceph version 14.2.11 (f7fdb2f52131f54b891a2ec99d8205561242cdaf) nautilus (stable) stderr: 1: (()+0xf630) [0x7f6006189630] stderr: 2: (gsignal()+0x37) [0x7f6004f7d387] stderr: 3: (abort()+0x148) [0x7f6004f7ea78] stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x199) [0x56366177c134] stderr: 5: (()+0x4d72ad) [0x56366177c2ad] stderr: 6: (BlueStore::~BlueStore()+0x7dd) [0x563661cf15fd] stderr: 7: (BlueStore::~BlueStore()+0x9) [0x563661cf1639] stderr: 8: (OSD::mkfs(CephContext*, ObjectStore*, uuid_d, int)+0x898) [0x563661839d28] stderr: 9: (main()+0x1501) [0x5636617e3481] stderr: 10: (__libc_start_main()+0xf5) [0x7f6004f69555] stderr: 11: (()+0x573c85) [0x563661818c85] stderr: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. stderr: -34> 2020-09-02 18:30:10.301 7f600913da80 -1 bluestore(/var/lib/ceph/osd/ceph-0/) _read_fsid unparsable uuid stderr: -5> 2020-09-02 18:30:10.303 7f600913da80 -1 rocksdb: Invalid argument: Can't parse option compaction_threads stderr: -3> 2020-09-02 18:30:10.558 7f600913da80 -1 bluestore(/var/lib/ceph/osd/ceph-0/) mkfs failed, (22) Invalid argument stderr: -2> 2020-09-02 18:30:10.558 7f600913da80 -1 OSD::mkfs: ObjectStore::mkfs failed with error (22) Invalid argument stderr: -1> 2020-09-02 18:30:10.567 7f600913da80 -1 /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/gigantic/release/14.2.11/rpm/el7/BUILD/ceph-14.2.11/src/os/bluestore/BlueStore.cc: In function 'virtual BlueStore::~BlueStore()' thread 7f600913da80 time 2020-09-02 18:30:10.559517 stderr: /home/jenkins-build/build/workspace/ceph-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/gigantic/release/14.2.11/rpm/el7/BUILD/ceph-14.2.11/src/os/bluestore/BlueStore.cc: 3944: FAILED ceph_assert(bluefs == __null) stderr: ceph version 14.2.11 (f7fdb2f52131f54b891a2ec99d8205561242cdaf) nautilus (stable) stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x14a) [0x56366177c0e5] stderr: 2: (()+0x4d72ad) [0x56366177c2ad] stderr: 3: (BlueStore::~BlueStore()+0x7dd) [0x563661cf15fd] stderr: 4: (BlueStore::~BlueStore()+0x9) [0x563661cf1639] stderr: 5: (OSD::mkfs(CephContext*, ObjectStore*, uuid_d, int)+0x898) [0x563661839d28] stderr: 6: (main()+0x1501) [0x5636617e3481] stderr: 7: (__libc_start_main()+0xf5) [0x7f6004f69555] stderr: 8: (()+0x573c85) [0x563661818c85] stderr: 0> 2020-09-02 18:30:10.574 7f600913da80 -1 *** Caught signal (Aborted) ** stderr: in thread 7f600913da80 thread_name:ceph-osd stderr: ceph version 14.2.11 (f7fdb2f52131f54b891a2ec99d8205561242cdaf) nautilus (stable) stderr: 1: (()+0xf630) [0x7f6006189630] stderr: 2: (gsignal()+0x37) [0x7f6004f7d387] stderr: 3: (abort()+0x148) [0x7f6004f7ea78] stderr: 4: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x199) [0x56366177c134] stderr: 5: (()+0x4d72ad) [0x56366177c2ad] stderr: 6: (BlueStore::~BlueStore()+0x7dd) [0x563661cf15fd] stderr: 7: (BlueStore::~BlueStore()+0x9) [0x563661cf1639] stderr: 8: (OSD::mkfs(CephContext*, ObjectStore*, uuid_d, int)+0x898) [0x563661839d28] stderr: 9: (main()+0x1501) [0x5636617e3481] stderr: 10: (__libc_start_main()+0xf5) [0x7f6004f69555] stderr: 11: (()+0x573c85) [0x563661818c85] stderr: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this. --> Was unable to complete a new OSD, will rollback changes Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring osd purge-new osd.0 --yes-i-really-mean-it stderr: 2020-09-02 18:30:10.730 7f4ea281a700 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,: (2) No such file or directory 2020-09-02 18:30:10.730 7f4ea281a700 -1 AuthRegistry(0x7f4e9c0656b8) no keyring found at /etc/ceph/ceph.client.bootstrap-osd.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,, disabling cephx stderr: purged osd.0 --> RuntimeError: Command failed with exit code 250: /usr/bin/ceph-osd --cluster ceph --osd-objectstore bluestore --mkfs -i 0 --monmap /var/lib/ceph/osd/ceph-0/activate.monmap --keyfile - --bluestore-block-wal-path /dev/optane/waldb --bluestore-block-db-path /dev/optane/rocksdb --osd-data /var/lib/ceph/osd/ceph-0/ --osd-uuid 2dd353ac-dcb8-48a8-88cc-cdf96b6ecfcf --setuser ceph --setgroup ceph
これすごく分かりづらいのですけど、bluestore_rocksdb_options の行を1行にまとめて書くと失敗してしまいます。
同じ内容でもカンマ区切りで改行してやるとcreateに成功します。
/etc/ceph/ceph.conf
bluestore_rocksdb_options = compression=kNoCompression,max_write_buffer_number=32,min_write_buffer_number_to_merge=2,recycle_log_file_num=32,compaction_style=kCompactionStyleLevel,write_buffer_size=67108864,target_file_size_base=67108864,max_background_compactions=31,level0_file_num_compaction_trigger=8,level0_slowdown_writes_trigger=32,level0_stop_writes_trigger=64,max_bytes_for_level_base=536870912,compaction_threads=32,max_bytes_for_level_multiplier=8,flusher_threads=8,compaction_readahead_size=2MB
/etc/ceph/ceph.conf
bluestore_rocksdb_options = compression=kNoCompression, max_write_buffer_number=32, min_write_buffer_number_to_merge=2, recycle_log_file_num=32, compaction_style=kCompactionStyleLevel, write_buffer_size=67108864, target_file_size_base=67108864, max_background_compactions=31, level0_file_num_compaction_trigger=8, level0_slowdown_writes_trigger=32, level0_stop_writes_trigger=64, max_bytes_for_level_base=536870912, compaction_threads=32, max_bytes_for_level_multiplier=8, flusher_threads=8, compaction_readahead_size=2MB