TTNetwork

The TTNetwork in TTPython abstracts over network interface types.

class ticktalkpython.Network.TTNetwork(ensemble_name, interface_type, sim=None, ip_addr=None, port=None, receiver_function=None)

This is a small wrapper class for the TTNetworkInterface that abstracts over network implementations. This also holds and manages the routing table, although the addresses within this will vary based on the actual network interface.

Parameters:

  • ensemble_name (string) – The (unique) name of this ensemble

  • interface_type (TTNetworkInterfaceType :param sim: If using a simulated form of network, this is the simulation environment. Defaults to None :type sim: simpy.Environment) – The type of TTNetworkInterface to be created. Depending on the value, the set of arguments will be used in different ways

  • ip (string) – If using a physical network interface that invokes the IP layer, include that as a string here in IPv4 format. Defaults to None

  • port (int) – The port to use for the network interface. Must be between 1024 and 65535 and must not be used by another other process on the machine. Defaults to None

  • receiver_function (function) – A callback to execute on whenever a message is received over the network. This should accept one input argument, i.e., the message received. This is generally a function that inserts the message into a shared queue.

add_route(recipient_ensemble_name, recipient_address)

Add an address to the network interface routing table

Parameters:

  • recipient_ensemble_name (string) – The name of the ensemble to send to; this must be unique among all ensembles in the system

  • recipient_address (string | TTEnsemble) – An identifier for the addres of the ensemble; specific to the network interface type. The format depends on the TTNetworkInterface implementation.

create_message(recipient_name, obj)

Create a network message for some object, to be sent to a specific ensemble. This will handle translating the object to send the destination into whatever the network interface expects

Parameters:

  • recipient_name (string) – The name of the ensemble the message should be sent to

  • obj (Any) – The payload of the message to send. It can be anything, so long as it can be serialized (using the python ‘pickle’ format)

Returns:

The message to be sent

Return type:

TTNetorkMessage or a child class

get_recipient_address(recipient_ensemble_name)

Retrieve the address from a routing table dictionary based on the name of a TTEnsemble

Parameters:

recipient_ensemble_name (string) – The name of the ensemble to send to; this must be unique among all ensembles in the system

send_message(message, ensure=False)

Send a message through the network interface; the message should have already been created with TTNetwork.create_message

Parameters:

  • message (TTNetworkMessage) – The message to send

  • ensure (bool) – Whether to attempt guaranteed delivery to the recipient ensemble

Returns:

None

update_route(recipient_ensemble_name, recipient_address)

Update an entry in the routing table

Parameters:

  • recipient_ensemble_name (string) – The name of the ensemble to send to; this must be unique among all ensembles in the system

  • recipient_address (string | TTEnsemble) – An identifier for the addres of the ensemble; specific to the network interface type

TTNetworkInterfaceUDP

A UDP network interface for TTPython. Generally, we decided to use UDP datagrams to avoid the overhead and maintenance of TCP connections. Best-efforts is an underlying pillar of TickTalk.

When incoming messages arrive to this Ensemble over the port we listen, the message payloads are provided to a callback function, which typically means inserting the message into a queue for another process/thread to respond to.

We include the option to use guaranteed delivery via a short handshaking and acknowledgement protocol, but this is not explicilty necessary. By default, that option is enabled for control-plane messages (graph setup, joining process, distributing SQs) whereas data-plane messages (sending tokens between SQs) are simply best-effort. This stack will automatically split large payloads into multiple datagrams, which are accounted for in this guaranteed-delivery process.

A single port is used for sending and receiving messages, partially so the handshaking process can be managed all in one place. Generally, our network implementation has one UDP interface created for outgoing messages and another one for incoming messages; these must use different ports.

class ticktalkpython.NetworkInterfaceUDP.TTAckPacket(message_id, sequence_num)

A wrapper object representing an ACK packet.

Parameters:

  • sequence_num (int) – The sequence number for the packet.

  • message_id (int) – the unique ID representing the TTNetworkMessage object contained within.

send(server_socket, address)
Parameters:

  • server_socket (Socket) – The given TTNetwork instance’s open server socket.

  • address (Address) – The recipient’s IP and port.

class ticktalkpython.NetworkInterfaceUDP.TTDataPacket(message_id=0, sequence_num=0, payload=None, ACKed=False)

A wrapper object representing a DATA packet.

Parameters:

  • sequence_num (int) – The sequence number for the packet.

  • payload (bytearray) – The data and header bytes to be sent in this packet.

  • ACKed (bool) – Indicates whether this message has been ACKed.

  • message_id (int) – the unique ID representing the TTNetworkMessage object contained within.

send(server_socket, address)
Parameters:

  • server_socket (Socket) – The given TTNetwork instance’s open server socket.

  • address (Address) – The recipient’s IP and port.

class ticktalkpython.NetworkInterfaceUDP.TTNetworkInterfaceUDP(ensembles=None, ip_addr='127.0.0.1', port=8225, timeout_ms=100, receiver_function=<function TTNetworkInterfaceUDP.<lambda>>)

In simulation, this represents the network that interconnects all ensembles in the system, providing a medium to exchange TTMessage objects. In practice, every ensemble will have an active instance of TTNetwork for use in sending and receiving messages. The TTNetwork constructor is non-blocking, as well as its send and receive operations. It represents a single port for the given ensemble for use in both sending and receiving messages.

Parameters:

  • ensembles (TTEnsemble) – The set of TTEnsemble objects involved in a TTPython simulation.

  • ip (str) – The IP public IP address of the current ensemble. Localhost can be used for simulation/testing, but the public IP must be set in order to receive external messages.

  • port (int) – The port for the network to send and receive on

  • timeout_ms (int) – the number of milliseconds to wait before resending packets

  • receiver_function (lambda TTNetworkMessage) – the function to be called once a TTNetworkMessage has been received.

cleanup()

Use to release OS resources allocated for the network

create_receiving_window(address, request_packet)

Create a TTSlidingWindow for a REQ. Will throw an error if a message ID is duplicated (which should be guaranteed not to occur based on the lifetime of the sender’s TTNetworkMessage object).

Parameters:

  • address (Address) – The address from which the message will be received

  • request_packet (TTReqPacket) – The TTReqPacket object received.

create_sending_window(message, after)

Used to create a TTSlidingWindow after receiving an ACK_START from the recipient. Will raise an Exception if a duplicate message ID is being sent, which will occur if the same TTNetworkMessage is being sent simultaneously to a single address.

Parameters:

  • message (TTNetworkMessage) – the TTNetworkMessage object to be sent.

  • after (lambda) – the function to be called once the message has been sent successfully in its entirety.

ensure_handshake(req_message, address)

Spawns a thread to ensure that a REQ to send a given TTNetworkMessage is ACKed by the recipient. The thread will wait on a boolean contained in the handshaking_threads dictionary, uniquely identified by the message’s ID and address.

Parameters:

  • req_message (TTReqPacket) – the TTReqPacket for the given message that is to be resent if not ACKed for

  • address (Address) – the address to which the request is being sent

get_receiving_window(address, message_id)

Get the TTSlidingWindow object for a message that is currently being received. Will return ‘None’ if no such message exists.

Parameters:

  • address (Address) – The address that the message is being received from

  • message_id (int) – the ID of the message being received

get_sending_window(dest_address, message_id)

Get the TTSlidingWindow object for a message that is currently being sent. Will return ‘None’ if no such message exists.

Parameters:

  • dest_address (Address) – The address that the requested window is sending to.

  • message_id (int) – The ID of the message that is currently being sent

handshake()

Wait for the packet timeout, and then resend the REQ for all messages that have not yet been handshaken for.

listen()

Receive packets at the specified port and public IP. Redirect them to the corresponding sending and receiving windows, creating new ones when necessary. This passes logic directly to the TTSlidingWindow objects concerned with each message.

On the receivers side, it is possible that message will come in that doesn’t match an active receiving window, and isn’t a REQ to create a new receiving window. These cases will occur quite often due to the nature of UDP; packets can be duplicated, and ACKs can be lost easily, so it’s possible for a receiver to finish receiving a message in full before the sender is aware. The sender might then have a timeout trigger for a packet before its ACK arrives, leading to duplicates, or a duplicate will simply occur without cause. These errors can be safely ignored, as seen below in the try-except blocks for WindowException.

send(message, after=<function TTNetworkInterfaceUDP.<lambda>>, ensure=False)

Create a sending window for a message, send a REQ for it to the sender, and then start a thread to ensure that the handshake completes successfully in case of packet loss. Returns immediately without blocking.

terminate_handshake(message_id, address)

Upon completion of a handshake for a given message ID and address, terminate the thread.

Parameters:

  • message_id (int) – The message ID that is being handshaken for

  • address – the Address to which the message will be sent

:type address Address

wrap_completion(msg, message_dict, message_id, callback)

When a message has been sent or received correctly, before calling the corresponding signalling function (after or receiver_function), ensure that the entry for that message’s sending/receiving window is deleted to avoid collision of message IDs.

Parameters:

  • msg (TTNetworkMessage) – The TTNetworkMessage object passed to the callback function on completion of sending/receiving.

  • message_dict (dict) – the dictionary containing the given message’s ID

  • message_id (int) – the ID of the message that has completed sending/receiving

  • callback (lambda) – the callback function to be executed

class ticktalkpython.NetworkInterfaceUDP.TTNetworkMessageUDP(recipient_ip='127.0.0.1', recipient_port=8080, payload_byte_array=None)

An encapsulation of a message to be sent into the network, consisting of a recipient, and a payload of arbitrary form. There are no restrictions on size of the payload; the Message is simply an encapsulation mechanism.

Parameters:

  • recipient_ip (str) – The ip of an ensemble to send a message to.

  • recipient_port (int) – The port that the recipient ensemble is listening on.

  • payload_byte_array (bytearray) – The actual data to send within the message. The format/type within the payload is arbitrary, depending ony on the message_type

generate_packets(window_size)

Splits the message’s payload_byte_array into a series of DATA packets and returns them as a list. The window_size parameter is used to enforce the constraint that the maximum sequence number of a packet must be more than twice the size of the window to ensure that collisions cannot occur.

Parameters:

window_size (int) – The size of the TTSlidingWindow in which this message will be sent.

get_address()

Return the destination address as an Address tuple object for use with Python’s UDP interface.

get_request()

Generates a TTRequestPacket instance for this message.

class ticktalkpython.NetworkInterfaceUDP.TTPacket(packet_type, message_id)

A wrapper object representing a payload containing a REQ, ACK, or DATA packet. When the send() method is called, the time in milliseconds is recorded for use in calculating if this packet has timed out.

Parameters:

  • packet_type (TTPacketType) – The type of TTPacket

  • message_id (int) – the unique ID representing the TTNetworkMessage object contained within.

send(server_socket, address)
Parameters:

  • server_socket (Socket) – The given TTNetwork instance’s open server socket.

  • address (Address) – The recipient’s IP and port.

class ticktalkpython.NetworkInterfaceUDP.TTPacketType(value, names=<not given>, *values, module=None, qualname=None, type=None, start=1, boundary=None)

Packets can either be a request to send (REQ), a data packet containing a portion of a message (DATA), or an acknowledgement that a particular data packet has been received (ACK). These are encoded as a single byte at the front of every packet.

class ticktalkpython.NetworkInterfaceUDP.TTRequestPacket(message_id, num_packets)

A wrapper object representing a REQ packet.

Parameters:

  • message_id (int) – the unique ID representing the TTNetworkMessage object contained within.

  • num_packets (int) – the number of packets for the receiver to expect for the given message.

class ticktalkpython.NetworkInterfaceUDP.TTSlidingWindow(window_size=1000, timeout_ms=100, completion_fn=<function TTSlidingWindow.<lambda>>, initializer=None)

TTSlidingWindow is an implementation of the Selective Repeat sliding window protocol, and is used to send AND receive every type of packet used in TTNetwork. Though it is contained and used by the TTNetwork’s primary message-receiving thread, it has its own timeout thread for the message that it represents. A receiving window is distinguished from a sending window based on the type of the initializer object; a receiving window is initialized by a TTRequestPacket, while a sending window is initialized by a TTNetworkMessage object.

Parameters:

  • window_size (int) – The number of packets that can be sent at once through this sliding window.

  • timeout_ms (int) – The number of milliseconds that the timeout thread waits for before checking for timed-out packets

  • completion_fn (lambda TTNetworkMessage) – A lambda to be called upon completion of the sending or receiving operation

  • initializer (TTRequestPacket | TTNetworkMessage) – A TTRequestPacket (receiving) or TTNetworkMessage (sending) used to determine if the window will be used for sending or receiving.

ack(ack_packet, server_socket, address)

The sender’s version of receiver(); this method takes a TTAckPacket object and marks the corresponding packet in the window as ACK-ed.

Parameters:

  • ack_packet (TTAckPacket) – an ACK for a given TTDataPacket

  • server_socket (Socket) – the parent TTNetwork object’s open server socket

  • address (Address) – the address from which the ACK was received

get_packet_index(sequence_number)

Given a sequence number, obtain the index in the current window where the packet with that sequence number should be/is stored. This assumes that the given sequence_number has been confirmed to be in the window with a call to is_in_window()

Parameters:

sequence_number (int) – the sequence number of a given packet

Returns:

the index of the packet in the window corresponding to the given sequence number

Return type:

int

is_in_window(sequence_number)

Indicates whether a given sequence number is valid for the current state of the sliding window; e.g. whether a packet with that number is being sent or can be received.

Parameters:

sequence_number (int) – the sequence number to be checked

Return type:

bool

print_window()

For debugging purposes only; prints the range of sequence numbers that are being sent/received at a given time. For accuracy and to avoid errors, this must be called AFTER securing the mutex. However, code to secure the mutex has not been added here because this method is most useful when called within the context of a method like ack() or receive(); both of which secure the mutex.

receive(data_packet, server_socket, address)

Given a received TTDataPacket object, add it to the window and send an ACK for it on the given server_socket to the given address. ACKs are sent for any packet, regardless if it is currently in the window or not. Restrictions on window sizes (sending=receiving) and sliding ensure that any packet that is not currently in the window has already been received and dealt with. This method triggers completion of the receiving process once the correct number of packets have been received, calling the completion_fn with the assembled message.

Parameters:

  • data_packet (TTDataPacket) – a packet of any sequence number that is part of the message to be received

  • server_socket (Socket) – the parent TTNetwork object’s open server socket

  • address (Address) – the address to which an ACK will be sent for the given packet

slide(server_socket, address)

Remove all ACK-ed packets/Received packets from the window and allow an equal number to be sent/received at the end of the window.

Parameters:

  • server_socket (Socket) – the parent TTNetwork’s open server socket

  • address (Address) – the address used to send packets that enter the window after sliding (if sender)

start(server_socket, address, timeout)

This method is called to active a sending window. It sends the first n packets specified by the window_size, and starts an independent thread to handle timeouts.

Parameters:

  • server_socket (Socket) – the parent TTNetwork’s open server socket

  • address – the address that the window will send to

  • timeout (int) – the number of milliseconds to wait before resending packets.

exception ticktalkpython.NetworkInterfaceUDP.WindowException
ticktalkpython.NetworkInterfaceUDP.convert_to_packet(payload)

Given a bytearray, produces a TTPacket instance for use by the receiver. Exceptions will be thrown if the given array doesn’t match the minimum lengths and formats required for a message. The value ‘None’ will be returned if the CRC check fails, in which case the TTNetwork should ignore the message, triggering the timeout on the sender’s side.

ticktalkpython.NetworkInterfaceUDP.convert_to_payload(packet)

Given a TTPacket instance, converts into a bytearray for sending over Python’s UDP interface.

ticktalkpython.NetworkInterfaceUDP.message(obj, recipient_ip='127.0.0.1', recipient_port=8080)

Provides a shorthand form for the TTNetworkMessage constructor, and handles serialization of the message’s contents.

TTNetworkInterfaceSim

A network interface for a simulated network.

This is the simplest form of network, in which the messages contain direct references to the other TTEnsemble Object; this is permissible when using simpy simulation environment, as the entire system runs in a single-threaded event loop such that memory references are safe enough to use between ensembles (Note that this strategy is only used in the network layer).

An extension of this interface can/should include a model for the network, such that network latency and packet loss can be injected and modulated to show how different system deployments or graph mappings impact system performance, such as end-to-end latency.

class ticktalkpython.NetworkInterfaceSim.TTNetworkInterfaceSim(sim)

Create the simulated version of the network interface, which includes a minimum implementation of a send and listen function.

In the simulated case, we maintain direct references to other ensembles such that having ensembles explicitly listen to the network interface is unnecessary and computationally wasteful (as it would mean creating another queue and process just to maintain an abstraction). In this way, there is no ‘receiver_function’ callback as an input argument

cleanup()

Use to release OS resources allocated for the network

listen()

Equivalent to receiving; sits in an infinite loop, waiting to receive messages over the network interface, and calls the receiver_function on the payload of the message, once arrived (and defragmented/stripped of headers, where applicable)

send(message, ensure=False)

Send a TTNetworkMessage through the interface. We skip the usual sender-receiver interface, and just inject it directly into the ensemble.

Parameters:

  • message (TTNetworkMessage) – The message to send to another ensemble

  • ensure (bool) – Whether the message should have guaranteed delivery. Currently unused, but inlcuding to satisfy the super class’s interface. Defaults to False

class ticktalkpython.NetworkInterfaceSim.TTNetworkMessageSim(recipient, payload)

A network message for the simulated environment