# Letting YDB's Topic Reader Close When the Stream Breaks

YDB's Python topic reader could hang on commit acknowledgements after a stream error. The merged fix stops the flush wait when the first error arrives.

Published: 2026-11-05
Updated: 2026-11-13

A reader cannot finish shutting down if it waits for acknowledgements from a stream that has already failed.

## Short answer

YDB's Python topic reader could hang in `close(flush=True)` when a broken stream still had uncommitted offsets pending. The flush waited for commit acknowledgements that could never arrive. The merged [PR #920](https://github.com/ydb-platform/ydb-python-sdk/pull/920) makes `ReaderStream.flush()` stop waiting when either the acknowledgements finish or the stream's first error is set. It closes [issue #679](https://github.com/ydb-platform/ydb-python-sdk/issues/679).

## The problem

The [original report](https://github.com/ydb-platform/ydb-python-sdk/issues/679) described a topic reader whose `close` method froze indefinitely. The reporter used SDK version 3.21.2, Python 3.12, and Ubuntu on x86_64, and supplied a producer-consumer reproduction. The expected behavior was for close to complete in a reasonable time.

## Root cause

`ReaderStream.flush()` collected the partition sessions' acknowledgement futures and awaited `asyncio.wait(futures)`. Once the stream broke, those acknowledgements could remain unresolved. Waiting for all of them prevented close from reaching stream teardown.

The [PR description](https://github.com/ydb-platform/ydb-python-sdk/pull/920) identifies the wait on an already-broken stream as the cause. The change races the acknowledgement wait against `self._first_error`.

## The fix

The [merged diff](https://github.com/ydb-platform/ydb-python-sdk/pull/920/files) replaces the unconditional acknowledgement wait with:

```python
if not futures:
    return

all_acks_received = asyncio.ensure_future(asyncio.wait(futures))
try:
    await asyncio.wait((all_acks_received, self._first_error), return_when=asyncio.FIRST_COMPLETED)
finally:
    all_acks_received.cancel()
```

An empty set of acknowledgement futures returns immediately. Otherwise, the first completed condition releases the wait. The `finally` block cancels the task wrapping the acknowledgement wait.

Two regression tests cover the failure path. `test_flush_stops_on_stream_error` checks that flush returns after a stream error while the pending acknowledgement remains unresolved, then verifies that close fails the waiter with `PublicTopicReaderPartitionExpiredError`. `test_close_with_flush_on_broken_stream_does_not_hang` sets a stream error before calling `close(True)` and checks the same waiter exception.

## By the numbers

- Repository: [ydb-platform/ydb-python-sdk](https://github.com/ydb-platform/ydb-python-sdk)
- Files changed: 3, including the implementation, tests, and changelog
- Lines: +36 / -3
- Commits: 3
- Opened: October 10, 2026. Merged: November 5, 2026
- Review: approved by a project member before merge

## How the review went

The public [coverage report](https://github.com/ydb-platform/ydb-python-sdk/pull/920#issuecomment-6095265948) states: "All modified and coverable lines are covered by tests." That is the report's statement, rather than an independent test run for this article.

The automated review also identified a coverage limitation: the regression test does not start a flush task and then inject a later stream error while that task is waiting. The merged tests exercise an already-broken stream. A project member [approved the PR](https://github.com/ydb-platform/ydb-python-sdk/pull/920#pullrequestreview-5782074512) on November 5, 2026, and the PR was merged that day.

## Takeaways

- **Wait for failure as well as success.** A shutdown flush needs an exit when its transport can no longer deliver acknowledgements.
- **Distinguish test scenarios.** An already-failed stream and a stream that fails during a wait are separate cases.
- **Keep teardown reachable.** Pending waiters can receive a meaningful exception only after the code reaches cleanup.

## FAQ

### Does this change wait for every acknowledgement after a stream error?

No. It stops the flush wait when `self._first_error` completes, so close can proceed with teardown. The regression tests expect pending waiters to fail with `PublicTopicReaderPartitionExpiredError`.

### Where can I inspect the change?

Read the [pull request](https://github.com/ydb-platform/ydb-python-sdk/pull/920), [diff and tests](https://github.com/ydb-platform/ydb-python-sdk/pull/920/files), and [original issue](https://github.com/ydb-platform/ydb-python-sdk/issues/679).

## The goal

A failed transport should not leave shutdown waiting forever for a successful response. This fix gives the topic reader's flush an error exit so close can finish cleaning up.

Source: https://prilog.ai/blogs/ydb-python-sdk-topic-reader-close-fix/
