Automated Test and Regression Environment

Continuous quality monitoring and improvement is a key need for any organization offering EDA tools or tool flows. Such organizations could be either EDA companies offering their EDA products or in-house CAD groups in semiconductor companies who are mandated to provision a robust tool chain to design teams. With ever shortening life span of tools and more frequent updates, there is lesser time available to undertake thorough regression testing. At the same time, with increasing complexity of tools, there is high probability of conceptual flaws in the tools that necessitates frequent benchmarking and insightful analysis on a large test database. Also, the quality of design flow is more important than quality of each tool, which implies that the tool flow need to be tested in a variety of configurations of each individual tool. There is also a need to best leverage the computing infrastructure available for such regression testing through optimal scheduling, distribution of jobs, collecting results and evaluating them with minimal manpower.

SoftJin’s Automated Test and Regression Environment is a first of its kind, versatile infrastructural framework that automates the Regression Testing of EDA tools and flows. Incorporating SoftJin’s deep domain knowledge of running regressions for in-house tools and third party tools and flows, this framework automates several commonly performed yet highly manual operations involved in Regression Set up, Execution and Reporting, thus addressing the various challenges associated with modern day Regression Testing as mentioned above. SoftJin’s Automated Test and Regression Environment include fully customizable modules for:

Organizing Regression Test Suite – This includes automated addition/modification of test cases on an ongoing basis into a Test Database. The test database stores information related to the input files associated with a test case, information on test case specific tool option, variety of attributes to label a test case and so on.
Organizing Tools – This includes automated addition/modification of tools into a Tool Database. The tool database stores information related to tool version and automated script to run the tool as part of a tool chain during a Regression Run.
Configuring and launching a Regression Run on a network of machines – The Regression Environment enables the user of Regression to easily query a set of test cases with desired attributes (from Test database) and run it through a desired tool flow (easily integrated from the Tools database). The Regression Run can be configured to be automatically distributed on a heterogeneous network of computing resources. The user can also specify the nature of the Regression reports and extent of Regression logs to be maintained. All these and more rich set of options can be set through an intuitive GUI form.
Analysis of Reports – The Regression Environment includes in-built report generation modules that extract a variety of meaningful analysis reports such as Regression Summary report, Runtime/Memory/Status report for each test case and each tool in the flow, Quality of Results (QoR) comparison with previous Regression Runs and so on. Customized Reports can be easily generated by defining complex pass/fail criteria for each tool and selecting the sub-set of QoR parameters of interest.

SoftJin Test and Regression Environment is based on completely data driven architecture which helps the user to tailor the regression infrastructure according to specific requirements, with minimal changes to the implementation code. The Environment is available on both Windows and Linux platform.

It supports parallel execution of test cases across distributed computer infrastructure consisting of heterogeneous platform including Solaris, Linux, Windows XP to reduce the regression run.

Structure for Automatic Test and Regression

SoftJin also offers EDA Software Lifecycle Management Services around the Automated Test and Regression environment. These services include setting up and customization of the Automated test and regression environment such as:

Customization of the Environment based upon the tools in the customer’s tool chain. This involves one time insertion of each tool into the tool database.
Test planning and organization of existing Test database – This involves insertion of all existing test cases into the Test database with appropriate attributes.
Customized Report Generation Set-up

Learn more about our Tool Quality Assurance and Validation Services