pypond.io package¶
Submodules¶
pypond.io.input module¶
Classes to handle pipeline input.
-
class
pypond.io.input.
Bounded
¶ Bases:
pypond.io.input.PipelineIn
For the pipeline - source of a fixed size - like a collection.
-
on_emit
()¶
-
start
()¶
-
stop
()¶
-
-
class
pypond.io.input.
PipelineIn
¶ Bases:
pypond.bases.Observable
For the pipeline - raise exceptions if an attempt is made to add heterogenous types.
-
class
pypond.io.input.
Stream
¶ Bases:
pypond.io.input.PipelineIn
For the pipeline - a source that has no container of its own.
-
add_event
(event)¶ Type check and event and emit it if we are running have have observers.
Parameters: event (Event) – Some Event class
-
events
()¶ Raise an exception - can’t iterate an unbounded source.
-
start
()¶
-
stop
()¶
-
pypond.io.output module¶
Objects to handle Pipeline output and event collection.
-
class
pypond.io.output.
CollectionOut
(pipeline, callback, options)¶ Bases:
pypond.io.output.PipelineOut
Output object for when processor results are being returned as a collection.
Parameters: -
flush
()¶ Flush the collector and mark the results_done = True in the pipeline if there is no longer an observer.
-
on_emit
(callback)¶ Sets the internal callback.
Parameters: callback (function or None) – Value to set the intenal _callback to.
-
-
class
pypond.io.output.
Collector
(options, on_trigger)¶ Bases:
pypond.bases.PypondBase
A Collector is used to accumulate events into multiple collections, based on potentially many strategies. In this current implementation a collection is partitioned based on the window that it falls in and the group it is part of.
Collections are emitted from this class to the supplied onTrigger callback.
Parameters: - options (Options) – A pipeline options instance
- on_trigger (function) – Callback to handle the emitted Collection
-
add_event
(event)¶ Add and event to the _collections dict and act accordingly depending on how _emit_on is set.
Parameters: event (Event) – An event. Raises: PipelineIOException
– Raised on bad args.
-
emit_collections
(collections)¶ Emit all of the collections to the trigger callback that was passed in by the Processor
Parameters: collections (dict) – A dict of string keys and Capsule objects containing the window_key, group_by_key and a Collection.
-
flush_collections
()¶ Emit the remaining collections.
-
class
pypond.io.output.
EventOut
(pipeline, callback=None, options=<pypond.util.Options object>)¶ Bases:
pypond.io.output.PipelineOut
Output object for when processor results are being returned as events.
Parameters: -
add_event
(event)¶ Add an event to the pipeline or callback.
Parameters: event (Event) – An event object
-
flush
()¶ Mark the results_done = True in the pipeline if there is no longer an observer.
-
on_emit
(callback)¶ Sets the internal callback.
Parameters: callback (function or None) – Value to set the intenal _callback to.
-
-
class
pypond.io.output.
PipelineOut
(pipeline)¶ Bases:
pypond.bases.PypondBase
Base class for pipeline output classes
Parameters: pipeline (Pipeline) – The Pipeline