Common Workflow Language

GitHub stars Gitter

The Common Workflow Language (CWL) is a specification for describing analysis workflows and tools that are portable and scalable across a variety of software and hardware environments, from workstations to cluster, cloud, and high performance computing (HPC) environments. CWL is designed to meet the needs of data-intensive science, such as Bioinformatics, Medical Imaging, Astronomy, Physics, and Chemistry.

CWL is developed by an informal, multi-vendor working group consisting of organizations and individuals interested enabling scientists to share data analysis workflows. The CWL project is on Github.

CWL builds on technologies such as JSON-LD for data modeling and Docker for portable runtime environments.

User Guide

The CWL user guide (v1.0) provides a gentle introduction to learning how to write CWL command line tool and workflow descriptions.

Specification

For developers and advanced users, the current CWL specification (v1.0) provides authoritative documentation of the execution of CWL documents. Links older drafts: draft-1, draft-2, draft-3

Citation

Please cite https://dx.doi.org/10.6084/m9.figshare.3115156.v2

Implementations

Software Description CWL support Platform support Maturity
cwltool Reference implementation of CWL Build Status Linux, OS X, local execution only Production
Arvados Distributed computing platform for data analysis on massive data sets. Using CWL on Arvados Build Status AWS, GCP, Azure, Slurm Production
Toil Toil is a workflow engine entirely written in Python. Build Status AWS, GCP, Azure, OpenStack, Grid Engine, Mesos Production
Apache Taverna Domain-independent Workflow Management System alpha Java Production
Galaxy Web-based platform for data intensive biomedical research. alpha Python Production
AWE Workflow and resource management system for bioinformatics data analysis. alpha Go Production
Funnel Use Google Genomics Pipeline API with CWL alpha GCP alpha
Rabix Bunny Reproducible Analyses for Bioinformatics alpha Java alpha

Repositories of CWL Tools and Workflows

Repository Description
Workflows repository Git repository of user contributed tools and workflows.
Dockstore tool registry An open platform for sharing Docker-based tools described with the Common Workflow Language used by the GA4GH.

Software for working with CWL

Software Description
cwltest CWL testing framework, automated testing of tools and workflows written with CWL
cwl-upgrader Upgrade CWL documents from draft-3 to v1.0
argparse2cwl Generate CWL CommandLineTool wrapper from Python programs that use argparse. Also supports click argument parser.
cwl2argparse Generate Python argparse code from CWL CommandLineTool description.
pypi2cwl Automatically run argparse2cwl on any package in PyPi
cwlavro Java classes for loading CWL documents

Support, Community and Contributing

The recommended place to ask a question about all things CWL is on Biostars.

Biostars CWL

If you are interested in learning more or contributing ideas or code, come chat with us on Gitter, check out #CommonWL on Twitter, join the mailing list common-workflow-language on Google Groups or fork the repository and send a pull request!

Your CWL Community Engineer, Michael R. Crusoe, publishes a blog about his work with updates at http://mrc.commonwl.org.

Code of Conduct

The CWL Project is dedicated to providing a harassment-free experience for everyone, regardless of gender, gender identity and expression, sexual orientation, disability, physical appearance, body size, age, race, or religion. We do not tolerate harassment of participants in any form.

This code of conduct applies to all CWL Project spaces, including the Google Group, the Gitter chat room, the Google Hangouts chats, both online and off. Anyone who violates this code of conduct may be sanctioned or expelled from these spaces at the discretion of the leadership team.

For more details, see our Code of Conduct

Participating Organizations

Individual Contributors

(Alphabetical)