Pipeline Architecture Implementation
building pipelined systems. I would be interested in your thoughts about the
two approaches.
First, I'm considering create an abstract Stage class that would accept an
input and output "stream". Each concrete Stage class would implement the
Stage's abstract methods which would do the actual work on the message. The
stage would have a single thead waiting for changes to the input stream.
Each stage would be chained together via the Stream objects. Once the stage
extracts a message from the input stream, then the message would be passed
down to the actual processing unit. The processing unit will populate an
output message. The output message would be inserted into the output stream
for further processing down stream.
or...
The second approach would still maintain a Stage class that would be the
actual message processing unit however this Stage class would not be waiting
on changes to an input stream. Therefore their would not be a thread per
stage. In place of the thread per Stage, I would create a pipeline controler
that would accept new messages into the pipeline. Once the controller
receives a message, the message would become a pipeline job that would be
assigned a thread from a thread pool. That thread would then execute each
stage of the pipeline.
I know there are lot's of details to consider with either approach, I'm
really curious about other experiences or possibly other approaches I could
consider.
Thanks,
Mike

