In many scientific fields such as bioinformatics, medical imaging, and
astronomy large quantities of data need to be
analyzed. This can involve large-scale and repetitive processes in long
pipelines of different tools — referred to as workflows. It can be very
time-consuming to run data for all these different tools by hand and convert
outputs to various formats to make them compatible with the next step.
Workflow management systems are designed to alleviate this problem by allowing
these workflows to be expressed formally and providing infrastructure to set up.
execute, and monitor them. This formal expression of workflows allows for
scientists to easily share and reuse them. Crucially they can also be used to
verify results of computation for published work. However there are many
competing [ways] for describing workflows which is a barrier to this aim.
Currently there are over a hundred different data analysis workflow systems
with no interoperability between them. The need has arisen to have a single
common standard and so the "Common Workflow Language" project was created: an
open standard designed to express workflows and their tooling in groups of
YAML structured text files.