Commonly used open source configuration management systems, such as
Puppet, Chef and CFEngine, allow for system configurations to be
expressed as scripts. A number of quality issues that may arise when
executing these scripts are identified. An automated quality assurance
service is proposed that identifies the presence of these issues by
automatically executing scripts across a range of environments. Test
results are automatically published to a format capable of being
consumed by script catalogues and social coding sites. This would serve
as an independent signal of script trustworthiness and quality to script
consumers and would allow developers to be made quickly aware of
quality issues. As a result, potential consumers of scripts can be
assured that a script is likely to work when applied to their particular
environment. Script developers can be notified of compatibility issues
and take steps to address them.