Context

Context is an information architecture that helps manage sets of data so they are stable (unchanging) and retrievable at any time. It is a vital component of batch processing, both at the end of day and especially for multiple times intraday.

A traditional batching scenario involving two systems that each supply two stable sets of data (for a total of four) for processing typically looks like this:

As straightforward as this sounds, the subtle issues are: A Context architecture revolves around a stateless service that permits participants to both create context definitions (name, metadata, and content) and ask to be vended same. This is not a central database of everything. It is a central locus of information that describes how a stable set of data can be vended, and the vending almost always will come from a separate technology. There are a few simple rules for Context:
  1. A context of data is immutable. Once created, a context of data, regardless of the underlying implementation that vends it, must always yield exactly the same set of data in exactly the same order. Note that a practical issue is emergency repairs or contexts created in error. This is accomodated by having a version number associated with each context and only the most recent version is ever vended.
  2. Context data lives forever. It is always possible to retrieve a context of data. Because the details of implementation are hidden from the consumer, the architecture can do clever things like switch to flatfiles for older material. Note that in general, files of data (not necessarily flat) are very attractive for this sort of need because they are fast, simple, and tend to "age" well.
With Context in place, participants outside the development and operations teams of Systems A, B, and C can observe create/consume dynamics among them. The Context concept scales to hundreds of systems with thousands of datasets.
Site copyright © 2013 Buzz Moschetti. All rights reserved