Failure to remove node ressources in simple python setting

General
1

Hey there, I keep encountering a warning that iceoryx2 was “Unable to remove node resources” when using iceoryx2 python bindings.

I tried this on a Ubuntu 24.04 using iceoryx2 v0.9.3 (also occured on current v0.9.999)

I managed to reduce my setup to the following example:

minimal_example.py 
import logging

import iceoryx2 as iox2

from typing import Optional
from ctypes import c_int

class Client:

    def __init__(self, app_name):
        self.app_name = app_name
        self.client = None

        # Debugging References
        self.node = None
        self.service_rr = None
        

    def connect(self):
        node = iox2.NodeBuilder.new().create(iox2.ServiceType.Ipc)
        service_rr = (
            node.service_builder(iox2.ServiceName.new(self.app_name)).request_response(c_int, c_int).open_or_create()
        )
        self.client = service_rr.client_builder().create()

        # (Don't) keep object references for debugging
        # self.node = node
        # self.service_rr = service_rr


def main():
    print("Start of main()")
    client = Client("Iceoryx2App")
    client.connect()
    print("End of main()")


if __name__ == "__main__":
    logging.basicConfig(level=logging.INFO)
    main()
    print("Done")

This should have the following output:

Start of main()
No config file was loaded, a config with default values will be used.
End of main()
WARNING:SharedNodeState {...}: Unable to remove node resources.
Done

To my understanding, something goes wrong during cleanup.
Interestingly, removing the comments for self.node and self.service_rr also removes the warning. However, documentation says that the Node going out of scope shouldnt be an issue due to SharedNode and SharedService references that the service and the client should, so it should be safe to discard the node + service ones the client was constructed?
Furthermore, the warning occurs after main() terminates, not as soon as node/service go out of scope.

I already mentioned the same warning in [another issue] ( Compatibility of WaitSet and wait operations with threads in python #1796 ), where I thought it was due to using threads.
This is not the case here.

Does anyone know how to avoid running into this warning?
The temporary storage at /tmp/iceoryx2/nodes indeed seems to be accumulating nodes that aren’t dropped properly, which I would like to avoid.

2

Can you set the log level to trace and post the output?

iox2.set_log_level_from_env_or(iox2.LogLevel.Trace)

3

Setting the log level to trace does not seem to change the output.
But using the logging.basicCOnfig(level=logging.DEBUG, which yields this output, which seems more detailed.
Particularly after the End of main() message, it starts to indicate insufficient permission and then the failure of removing the node folder.

Console output

Start of main()
DEBUG:FileBuilder { file_path: FilePath { value: StaticString<255> { len: 20, data: "config/iceoryx2.toml" } }, access_mode: Read, permission: Permission(448), has_ownership: false, owner: None, group: None, truncate_size: None, creation_mode: None }:Unable to open file since it does not exist.
DEBUG:Config { global: Global { root_path: Path { value: StaticString<255> { len: 14, data: "/tmp/iceoryx2/" } }, prefix: FileName { value: StaticString<255> { len: 5, data: "iox2_" } }, service: Service { directory: Path { value: StaticString<255> { len: 8, data: "services" } }, data_segment_suffix: FileName { value: StaticString<255> { len: 5, data: ".data" } }, static_config_storage_suffix: FileName { value: StaticString<255> { len: 8, data: ".service" } }, dynamic_config_storage_suffix: FileName { value: StaticString<255> { len: 8, data: ".dynamic" } }, connection_suffix: FileName { value: StaticString<255> { len: 11, data: ".connection" } }, event_connection_suffix: FileName { value: StaticString<255> { len: 6, data: ".event" } }, blackboard_mgmt_suffix: FileName { value: StaticString<255> { len: 16, data: ".blackboard_mgmt" } }, blackboard_data_suffix: FileName { value: StaticString<255> { len: 16, data: ".blackboard_data" } }, cleanup_dead_nodes_on_open: true }, node: Node { directory: Path { value: StaticString<255> { len: 5, data: "nodes" } }, monitor_suffix: FileName { value: StaticString<255> { len: 13, data: ".node_monitor" } }, static_config_suffix: FileName { value: StaticString<255> { len: 8, data: ".details" } }, service_tag_suffix: FileName { value: StaticString<255> { len: 12, data: ".service_tag" } }, port_tag_suffix: FileName { value: StaticString<255> { len: 9, data: ".port_tag" } }, global_mgmt_suffix: FileName { value: StaticString<255> { len: 12, data: ".global_mgmt" } }, cleanup_dead_nodes_on_creation: true, cleanup_dead_nodes_on_destruction: true }, creation_timeout: 500ms }, defaults: Defaults { publish_subscribe: PublishSubscribe { max_subscribers: 8, max_publishers: 2, max_nodes: 20, subscriber_max_buffer_size: 2, subscriber_max_borrowed_samples: 2, publisher_max_loaned_samples: 2, publisher_history_size: 0, enable_safe_overflow: true, backpressure_strategy: RetryUntilDelivered, subscriber_expired_connection_buffer: 128, publisher_allocation_strategy: Static }, event: Event { max_listeners: 16, max_notifiers: 16, max_nodes: 36, event_id_max_value: 255, deadline: None, notifier_created_event: None, notifier_dropped_event: None, notifier_dead_event: None }, request_response: RequestResonse { enable_safe_overflow_for_requests: true, enable_safe_overflow_for_responses: true, max_active_requests_per_client: 4, max_response_buffer_size: 2, max_servers: 2, max_clients: 8, max_nodes: 20, max_borrowed_responses_per_pending_response: 2, max_loaned_requests: 2, server_max_loaned_responses_per_request: 2, client_backpressure_strategy: RetryUntilDelivered, server_backpressure_strategy: RetryUntilDelivered, client_expired_connection_buffer: 128, enable_fire_and_forget_requests: true, server_expired_connection_buffer: 128, client_allocation_strategy: Static, server_allocation_strategy: Static }, blackboard: Blackboard { max_readers: 8, max_nodes: 20 } } }:Failed to create config since the config file "config/iceoryx2.toml" does not exist.
DEBUG:"Config.global_config()":No config file found at "config/iceoryx2.toml"
DEBUG:FileBuilder { file_path: FilePath { value: StaticString<255> { len: 49, data: "/home/tobias_dietz/.config/iceoryx2/iceoryx2.toml" } }, access_mode: Read, permission: Permission(448), has_ownership: false, owner: None, group: None, truncate_size: None, creation_mode: None }:Unable to open file since it does not exist.
DEBUG:Config { global: Global { root_path: Path { value: StaticString<255> { len: 14, data: "/tmp/iceoryx2/" } }, prefix: FileName { value: StaticString<255> { len: 5, data: "iox2_" } }, service: Service { directory: Path { value: StaticString<255> { len: 8, data: "services" } }, data_segment_suffix: FileName { value: StaticString<255> { len: 5, data: ".data" } }, static_config_storage_suffix: FileName { value: StaticString<255> { len: 8, data: ".service" } }, dynamic_config_storage_suffix: FileName { value: StaticString<255> { len: 8, data: ".dynamic" } }, connection_suffix: FileName { value: StaticString<255> { len: 11, data: ".connection" } }, event_connection_suffix: FileName { value: StaticString<255> { len: 6, data: ".event" } }, blackboard_mgmt_suffix: FileName { value: StaticString<255> { len: 16, data: ".blackboard_mgmt" } }, blackboard_data_suffix: FileName { value: StaticString<255> { len: 16, data: ".blackboard_data" } }, cleanup_dead_nodes_on_open: true }, node: Node { directory: Path { value: StaticString<255> { len: 5, data: "nodes" } }, monitor_suffix: FileName { value: StaticString<255> { len: 13, data: ".node_monitor" } }, static_config_suffix: FileName { value: StaticString<255> { len: 8, data: ".details" } }, service_tag_suffix: FileName { value: StaticString<255> { len: 12, data: ".service_tag" } }, port_tag_suffix: FileName { value: StaticString<255> { len: 9, data: ".port_tag" } }, global_mgmt_suffix: FileName { value: StaticString<255> { len: 12, data: ".global_mgmt" } }, cleanup_dead_nodes_on_creation: true, cleanup_dead_nodes_on_destruction: true }, creation_timeout: 500ms }, defaults: Defaults { publish_subscribe: PublishSubscribe { max_subscribers: 8, max_publishers: 2, max_nodes: 20, subscriber_max_buffer_size: 2, subscriber_max_borrowed_samples: 2, publisher_max_loaned_samples: 2, publisher_history_size: 0, enable_safe_overflow: true, backpressure_strategy: RetryUntilDelivered, subscriber_expired_connection_buffer: 128, publisher_allocation_strategy: Static }, event: Event { max_listeners: 16, max_notifiers: 16, max_nodes: 36, event_id_max_value: 255, deadline: None, notifier_created_event: None, notifier_dropped_event: None, notifier_dead_event: None }, request_response: RequestResonse { enable_safe_overflow_for_requests: true, enable_safe_overflow_for_responses: true, max_active_requests_per_client: 4, max_response_buffer_size: 2, max_servers: 2, max_clients: 8, max_nodes: 20, max_borrowed_responses_per_pending_response: 2, max_loaned_requests: 2, server_max_loaned_responses_per_request: 2, client_backpressure_strategy: RetryUntilDelivered, server_backpressure_strategy: RetryUntilDelivered, client_expired_connection_buffer: 128, enable_fire_and_forget_requests: true, server_expired_connection_buffer: 128, client_allocation_strategy: Static, server_allocation_strategy: Static }, blackboard: Blackboard { max_readers: 8, max_nodes: 20 } } }:Failed to create config since the config file "/home/tobias_dietz/.config/iceoryx2/iceoryx2.toml" does not exist.
DEBUG:"Config.global_config()":No config file found at "/home/tobias_dietz/.config/iceoryx2/iceoryx2.toml"
DEBUG:FileBuilder { file_path: FilePath { value: StaticString<255> { len: 27, data: "/etc/iceoryx2/iceoryx2.toml" } }, access_mode: Read, permission: Permission(448), has_ownership: false, owner: None, group: None, truncate_size: None, creation_mode: None }:Unable to open file since it does not exist.
DEBUG:Config { global: Global { root_path: Path { value: StaticString<255> { len: 14, data: "/tmp/iceoryx2/" } }, prefix: FileName { value: StaticString<255> { len: 5, data: "iox2_" } }, service: Service { directory: Path { value: StaticString<255> { len: 8, data: "services" } }, data_segment_suffix: FileName { value: StaticString<255> { len: 5, data: ".data" } }, static_config_storage_suffix: FileName { value: StaticString<255> { len: 8, data: ".service" } }, dynamic_config_storage_suffix: FileName { value: StaticString<255> { len: 8, data: ".dynamic" } }, connection_suffix: FileName { value: StaticString<255> { len: 11, data: ".connection" } }, event_connection_suffix: FileName { value: StaticString<255> { len: 6, data: ".event" } }, blackboard_mgmt_suffix: FileName { value: StaticString<255> { len: 16, data: ".blackboard_mgmt" } }, blackboard_data_suffix: FileName { value: StaticString<255> { len: 16, data: ".blackboard_data" } }, cleanup_dead_nodes_on_open: true }, node: Node { directory: Path { value: StaticString<255> { len: 5, data: "nodes" } }, monitor_suffix: FileName { value: StaticString<255> { len: 13, data: ".node_monitor" } }, static_config_suffix: FileName { value: StaticString<255> { len: 8, data: ".details" } }, service_tag_suffix: FileName { value: StaticString<255> { len: 12, data: ".service_tag" } }, port_tag_suffix: FileName { value: StaticString<255> { len: 9, data: ".port_tag" } }, global_mgmt_suffix: FileName { value: StaticString<255> { len: 12, data: ".global_mgmt" } }, cleanup_dead_nodes_on_creation: true, cleanup_dead_nodes_on_destruction: true }, creation_timeout: 500ms }, defaults: Defaults { publish_subscribe: PublishSubscribe { max_subscribers: 8, max_publishers: 2, max_nodes: 20, subscriber_max_buffer_size: 2, subscriber_max_borrowed_samples: 2, publisher_max_loaned_samples: 2, publisher_history_size: 0, enable_safe_overflow: true, backpressure_strategy: RetryUntilDelivered, subscriber_expired_connection_buffer: 128, publisher_allocation_strategy: Static }, event: Event { max_listeners: 16, max_notifiers: 16, max_nodes: 36, event_id_max_value: 255, deadline: None, notifier_created_event: None, notifier_dropped_event: None, notifier_dead_event: None }, request_response: RequestResonse { enable_safe_overflow_for_requests: true, enable_safe_overflow_for_responses: true, max_active_requests_per_client: 4, max_response_buffer_size: 2, max_servers: 2, max_clients: 8, max_nodes: 20, max_borrowed_responses_per_pending_response: 2, max_loaned_requests: 2, server_max_loaned_responses_per_request: 2, client_backpressure_strategy: RetryUntilDelivered, server_backpressure_strategy: RetryUntilDelivered, client_expired_connection_buffer: 128, enable_fire_and_forget_requests: true, server_expired_connection_buffer: 128, client_allocation_strategy: Static, server_allocation_strategy: Static }, blackboard: Blackboard { max_readers: 8, max_nodes: 20 } } }:Failed to create config since the config file "/etc/iceoryx2/iceoryx2.toml" does not exist.
DEBUG:"Config.global_config()":No config file found at "/etc/iceoryx2/iceoryx2.toml"
WARNING:"Config.global_config()":No config file was loaded, a config with default values will be used.
DEBUG:FileBuilder { file_path: FilePath { value: StaticString<255> { len: 76, data: "/tmp/iceoryx2/nodes/iox2_400579559570848054441788050017.node_monitor_context" } }, access_mode: Write, permission: Permission(448), has_ownership: false, owner: None, group: None, truncate_size: None, creation_mode: None }:Unable to open file due to insufficient permissions.
DEBUG:ProcessMonitor { state_path: FilePath { value: StaticString<255> { len: 68, data: "/tmp/iceoryx2/nodes/iox2_400579559570848054441788050017.node_monitor" } }, owner_lock_path: FilePath { value: StaticString<255> { len: 79, data: "/tmp/iceoryx2/nodes/iox2_400579559570848054441788050017.node_monitor_owner_lock" } }, context_path: FilePath { value: StaticString<255> { len: 76, data: "/tmp/iceoryx2/nodes/iox2_400579559570848054441788050017.node_monitor_context" } } }:Unable to open ProcessMonitor state file "/tmp/iceoryx2/nodes/iox2_400579559570848054441788050017.node_monitor_context" with access mode Write due to insufficient permissions.
End of main()
DEBUG:FileBuilder { file_path: FilePath { value: StaticString<255> { len: 76, data: "/tmp/iceoryx2/nodes/iox2_400579559570848054441788050017.node_monitor_context" } }, access_mode: Write, permission: Permission(448), has_ownership: false, owner: None, group: None, truncate_size: None, creation_mode: None }:Unable to open file due to insufficient permissions.
DEBUG:ProcessMonitor { state_path: FilePath { value: StaticString<255> { len: 68, data: "/tmp/iceoryx2/nodes/iox2_400579559570848054441788050017.node_monitor" } }, owner_lock_path: FilePath { value: StaticString<255> { len: 79, data: "/tmp/iceoryx2/nodes/iox2_400579559570848054441788050017.node_monitor_owner_lock" } }, context_path: FilePath { value: StaticString<255> { len: 76, data: "/tmp/iceoryx2/nodes/iox2_400579559570848054441788050017.node_monitor_context" } } }:Unable to open ProcessMonitor state file "/tmp/iceoryx2/nodes/iox2_400579559570848054441788050017.node_monitor_context" with access mode Write due to insufficient permissions.
DEBUG:"Directory.remove_empty()":Unable to remove empty directory "/tmp/iceoryx2/nodes/400579559570848054441788050017" since the directory is not empty or there are hard links pointing to the directory.
DEBUG:"remove_path_hint(Path { value: StaticString<255> { len: 50, data: "/tmp/iceoryx2/nodes/400579559570848054441788050017" } })":Unable to remove path hint due to an internal error (NotEmptyOrHardLinksPointingToTheDirectory).
DEBUG:"remove_node_details_directory(Config { global: Global { root_path: Path { value: StaticString<255> { len: 14, data: "/tmp/iceoryx2/" } }, prefix: FileName { value: StaticString<255> { len: 5, data: "iox2_" } }, service: Service { directory: Path { value: StaticString<255> { len: 8, data: "services" } }, data_segment_suffix: FileName { value: StaticString<255> { len: 5, data: ".data" } }, static_config_storage_suffix: FileName { value: StaticString<255> { len: 8, data: ".service" } }, dynamic_config_storage_suffix: FileName { value: StaticString<255> { len: 8, data: ".dynamic" } }, connection_suffix: FileName { value: StaticString<255> { len: 11, data: ".connection" } }, event_connection_suffix: FileName { value: StaticString<255> { len: 6, data: ".event" } }, blackboard_mgmt_suffix: FileName { value: StaticString<255> { len: 16, data: ".blackboard_mgmt" } }, blackboard_data_suffix: FileName { value: StaticString<255> { len: 16, data: ".blackboard_data" } }, cleanup_dead_nodes_on_open: true }, node: Node { directory: Path { value: StaticString<255> { len: 5, data: "nodes" } }, monitor_suffix: FileName { value: StaticString<255> { len: 13, data: ".node_monitor" } }, static_config_suffix: FileName { value: StaticString<255> { len: 8, data: ".details" } }, service_tag_suffix: FileName { value: StaticString<255> { len: 12, data: ".service_tag" } }, port_tag_suffix: FileName { value: StaticString<255> { len: 9, data: ".port_tag" } }, global_mgmt_suffix: FileName { value: StaticString<255> { len: 12, data: ".global_mgmt" } }, cleanup_dead_nodes_on_creation: true, cleanup_dead_nodes_on_destruction: true }, creation_timeout: 500ms }, defaults: Defaults { publish_subscribe: PublishSubscribe { max_subscribers: 8, max_publishers: 2, max_nodes: 20, subscriber_max_buffer_size: 2, subscriber_max_borrowed_samples: 2, publisher_max_loaned_samples: 2, publisher_history_size: 0, enable_safe_overflow: true, backpressure_strategy: RetryUntilDelivered, subscriber_expired_connection_buffer: 128, publisher_allocation_strategy: Static }, event: Event { max_listeners: 16, max_notifiers: 16, max_nodes: 36, event_id_max_value: 255, deadline: None, notifier_created_event: None, notifier_dropped_event: None, notifier_dead_event: None }, request_response: RequestResonse { enable_safe_overflow_for_requests: true, enable_safe_overflow_for_responses: true, max_active_requests_per_client: 4, max_response_buffer_size: 2, max_servers: 2, max_clients: 8, max_nodes: 20, max_borrowed_responses_per_pending_response: 2, max_loaned_requests: 2, server_max_loaned_responses_per_request: 2, client_backpressure_strategy: RetryUntilDelivered, server_backpressure_strategy: RetryUntilDelivered, client_expired_connection_buffer: 128, enable_fire_and_forget_requests: true, server_expired_connection_buffer: 128, client_allocation_strategy: Static, server_allocation_strategy: Static }, blackboard: Blackboard { max_readers: 8, max_nodes: 20 } } }, UniqueNodeId(UniqueSystemId { value: 400579559570848054441788050017, pid: 2657, creation_time: Time { clock_type: Monotonic, seconds: 183, nanoseconds: 240624957 } }))":Unable to remove node details directory due to an internal error.
WARNING:SharedNodeState { id: UniqueNodeId(UniqueSystemId { value: 400579559570848054441788050017, pid: 2657, creation_time: Time { clock_type: Monotonic, seconds: 183, nanoseconds: 240624957 } }), details: NodeDetails { executable: FileName { value: StaticString<255> { len: 10, data: "python3.12" } }, name: NodeName { value: StaticString<128> { len: 0, data: "" } }, config: Config { global: Global { root_path: Path { value: StaticString<255> { len: 14, data: "/tmp/iceoryx2/" } }, prefix: FileName { value: StaticString<255> { len: 5, data: "iox2_" } }, service: Service { directory: Path { value: StaticString<255> { len: 8, data: "services" } }, data_segment_suffix: FileName { value: StaticString<255> { len: 5, data: ".data" } }, static_config_storage_suffix: FileName { value: StaticString<255> { len: 8, data: ".service" } }, dynamic_config_storage_suffix: FileName { value: StaticString<255> { len: 8, data: ".dynamic" } }, connection_suffix: FileName { value: StaticString<255> { len: 11, data: ".connection" } }, event_connection_suffix: FileName { value: StaticString<255> { len: 6, data: ".event" } }, blackboard_mgmt_suffix: FileName { value: StaticString<255> { len: 16, data: ".blackboard_mgmt" } }, blackboard_data_suffix: FileName { value: StaticString<255> { len: 16, data: ".blackboard_data" } }, cleanup_dead_nodes_on_open: true }, node: Node { directory: Path { value: StaticString<255> { len: 5, data: "nodes" } }, monitor_suffix: FileName { value: StaticString<255> { len: 13, data: ".node_monitor" } }, static_config_suffix: FileName { value: StaticString<255> { len: 8, data: ".details" } }, service_tag_suffix: FileName { value: StaticString<255> { len: 12, data: ".service_tag" } }, port_tag_suffix: FileName { value: StaticString<255> { len: 9, data: ".port_tag" } }, global_mgmt_suffix: FileName { value: StaticString<255> { len: 12, data: ".global_mgmt" } }, cleanup_dead_nodes_on_creation: true, cleanup_dead_nodes_on_destruction: true }, creation_timeout: 500ms }, defaults: Defaults { publish_subscribe: PublishSubscribe { max_subscribers: 8, max_publishers: 2, max_nodes: 20, subscriber_max_buffer_size: 2, subscriber_max_borrowed_samples: 2, publisher_max_loaned_samples: 2, publisher_history_size: 0, enable_safe_overflow: true, backpressure_strategy: RetryUntilDelivered, subscriber_expired_connection_buffer: 128, publisher_allocation_strategy: Static }, event: Event { max_listeners: 16, max_notifiers: 16, max_nodes: 36, event_id_max_value: 255, deadline: None, notifier_created_event: None, notifier_dropped_event: None, notifier_dead_event: None }, request_response: RequestResonse { enable_safe_overflow_for_requests: true, enable_safe_overflow_for_responses: true, max_active_requests_per_client: 4, max_response_buffer_size: 2, max_servers: 2, max_clients: 8, max_nodes: 20, max_borrowed_responses_per_pending_response: 2, max_loaned_requests: 2, server_max_loaned_responses_per_request: 2, client_backpressure_strategy: RetryUntilDelivered, server_backpressure_strategy: RetryUntilDelivered, client_expired_connection_buffer: 128, enable_fire_and_forget_requests: true, server_expired_connection_buffer: 128, client_allocation_strategy: Static, server_allocation_strategy: Static }, blackboard: Blackboard { max_readers: 8, max_nodes: 20 } } } }, monitoring_token: UnsafeCell(UnsafeCell { .. }), registered_services: RegisteredServices { handle: MutexHandle { handle: HandleStorage<libc.unix.linux_like.linux.pthread_mutex_t> { is_interprocess_capable: false, is_initialized: true }, clock_type: UnsafeCell(UnsafeCell { .. }), value: UnsafeCell(UnsafeCell { .. }) } }, signal_handling_mode: HandleTerminationRequests, details_storage: Storage { name: FileName { value: StaticString<255> { len: 4, data: "node" } }, config: Configuration { path: Path { value: StaticString<255> { len: 50, data: "/tmp/iceoryx2/nodes/400579559570848054441788050017" } }, suffix: FileName { value: StaticString<255> { len: 8, data: ".details" } }, prefix: FileName { value: StaticString<255> { len: 5, data: "iox2_" } } }, has_ownership: AtomicBool(false), file: File { path: Some(FilePath { value: StaticString<255> { len: 68, data: "/tmp/iceoryx2/nodes/400579559570848054441788050017/iox2_node.details" } }), file_descriptor: FileDescriptor { value: 3, is_owned: true }, access_mode: ReadWrite, has_ownership: AtomicBool(false) }, len: 1989 } }:Unable to remove node resources.
Done

4

I suspect that some garbage collection causes the cleanup order to be different/random then it would be in languages like C++.

You are right, we have created a shared node state and shared service state that should handle exactly this use case.
Could you try to call self.client.delete() before leaving main? This would explicitly remove the client and should solve the issue temporarily.

If this is the case, we would have a better understanding of where this issue is caused. In the meantime, I will do some digging and try to understand the cleanup order and see if we have mixed something up there.

5

Whoopsie, I found it.

We stated that the shared node state / shared service state is part of the ports and services, but it seems it is not. The good thing is, this can be fixed quickly. I will write some tests and then your problem should be gone at the end of this week.

This was never an issue since languages like C++ and Rust have a clearly defined destruction order where such an issue should be never encountered.

6

I was mistaken. The shared state was just well hidden :confused:

I wrote unit tests for all services and ports and they proof that at least to underlying implementation and Rust follows the stated contract. So it must be an issue in the python bindings.

Could you start your application again with trace logging (logging.basicConfig(level=logging.TRACE)) and provide us the full log output. The trace log should contain the full destruction order of all resources.

7

The python logging library unfortunately doesn’t have a logging.TRACE level.

Further, I have tried setting the iox2.LogLevel.Trace in 3 ways:

  1. Directly in the code with iox2.set_log_level(iox2.LogLevel.Trace)
  2. Directly in the code iox2.set_log_level_from_env_or(iox2.LogLevel.Trace)
  3. Running export IOX2_LOG_LEVEL=Tracein the shell and then running iox2.set_log_level_from_env_or_default()

None of these changed the output however. I also removed the logging library to ensure it doesn’t interfere but still couldn’t access the iox2 logging.
The only way to access the debug logs was through setting logging.Debug

I also couldn’t find any tests for the log level under iceoryx2-ffi/python/tests.
The examples only use LogLevel.Info and setting it to LogLevel.Tracethere also didn’t produce trace level output.

Am I missing something or is this another issue altogether?
Are you able to reproduce the issue with the given example?

9

The bug is fixed, and I added some more unit tests that also verify that the underlying resources are removed: [#1971] destruction order by elfenpiff · Pull Request #1972 · eclipse-iceoryx/iceoryx2 · GitHub

If you check out current main, the issue should be solved on your end.