добрый день, за последние пару дней в логах посыпались постоянные "Session expired" (wc-l по err-логу 36000 ) и таблицы уходят в readonly и возвращаются.
При этом ЗК живой вполне.
куда можно копать?
2020.08.11 08:36:22.669140 [ 8046 ] {} <Error> schema.tablename: void DB::StorageReplicatedMergeTree::mutationsUpdatingTask(): Code: 999, e.displayText() = Coordination::Exception: Session expired (Session expired), Stack trace (when copying this message, always include the lines below):
0. Poco::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x1051b3a0 in /usr/bin/clickhouse
1. DB::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0x8f5172d in /usr/bin/clickhouse
2. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int, int) @ 0xdeef818 in /usr/bin/clickhouse
3. Coordination::Exception::Exception(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int) @ 0xdeefe42 in /usr/bin/clickhouse
4. ? @ 0xdf24221 in /usr/bin/clickhouse
5. Coordination::ZooKeeper::list(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::function<void (Coordination::ListResponse const&)>, std::__1::function<void (Coordination::WatchResponse const&)>) @ 0xdf1f68d in /usr/bin/clickhouse
6. zkutil::ZooKeeper::getChildrenImpl(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > > >&, Coordination::Stat*, std::__1::function<void (Coordination::WatchResponse const&)>) @ 0xdef7bd7 in /usr/bin/clickhouse
7. zkutil::ZooKeeper::getChildrenWatch(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Coordination::Stat*, std::__1::function<void (Coordination::WatchResponse const&)>) @ 0xdefc01f in /usr/bin/clickhouse
8. DB::ReplicatedMergeTreeQueue::updateMutations(std::__1::shared_ptr<zkutil::ZooKeeper>, std::__1::function<void (Coordination::WatchResponse const&)>) @ 0xdac42d8 in /usr/bin/clickhouse
9. DB::StorageReplicatedMergeTree::mutationsUpdatingTask() @ 0xd7291e5 in /usr/bin/clickhouse
10. DB::BackgroundSchedulePoolTaskInfo::execute() @ 0xcfe9415 in /usr/bin/clickhouse
11. DB::BackgroundSchedulePool::threadFunction() @ 0xcfe9a32 in /usr/bin/clickhouse
12. ? @ 0xcfe9b70 in /usr/bin/clickhouse
13. ThreadPoolImpl<std::__1::thread>::worker(std::__1::__list_iterator<std::__1::thread, void*>) @ 0x8f757e7 in /usr/bin/clickhouse
14. ? @ 0x8f73c33 in /usr/bin/clickhouse
15. start_thread @ 0x7ea5 in /usr/lib64/libpthread-2.17.so
16. __clone @ 0xfe8cd in /usr/lib64/libc-2.17.so
(version 20.3.11.97 (official build))
