MES Test Manager® FAQ
Here you can find a list of frequently asked questions about MES Test Manager and some useful answers. If you still can't find what you're looking for, please feel free to contact us directly.
1. System Requirements, Installation, Setup
Which Matlab / Simulink releases are supported?
Matlab R2007b and more recent versions, including R2007b, R2008a/b, R2009a/b, R2010a/b, R2011a/b, R2012a/b, R2013a/b, R2014a/b (and provisionally future releases).
Which operating systems and test environments are supported?
Matlab running on Windows, Linux, Mac, UNIX, etc. MTest is integrated into Matlab and is completely written in m. We have therefore designed MTest to run on all platforms that are supported by Matlab / Simulink. MTest is proven in use under Windows XP, Windows 7, Windows 8 and Mac OS X.
How long does it take to install MTest?
It takes approximately 10 minutes to read chapter 2 of the MTest User Guide. Then another 4 minutes to install the program. This involves copying and unpacking the zip file, and incorporating an installation path in the Matlab path so MTest can be launched from any directory. If you want to see the installation process in action, take a look at our Installation Tutorial in the videos section of this website.
Is the MES Test Manager available in a network version?
An extra network installation version is not necessary, because the standard version can be used either as a local or a network installation. It can be used by multiple users at one time or by one user working on multiple projects at the same time. User-specific data is saved under individual user names, whereas project-specific data is stored in project directories.
2. Getting Started
How long does it take to learn how to work with MTest ?
We recommend beginning with the Quick Start chapter of the MTest User Guide. Working through this chapter takes around an hour and will teach you all the basic functions of MTest. Take also a look at our Quick Start Tutorials in the videos section of this website.
You will need another two hours or so to learn batch testing. And around two hours for learning how to import the various data formats. Understanding how to perform test definitions with the CTE (optional) takes a little longer (around one day for most functions).
Is a minimum knowledge of Matlab and the model necessary?
Yes and no. Using MTest requires no prior knowledge of Matlab. However, it helps to have a basic understanding of working with Matlab, and Simulink in particular. The models under test have been created in Simulink/TargetLink. To test these properly, you should really have a basic understanding of these models (even when carrying out black box testing).
3. Model I/O
Which model parts can be tested?
Any number of model subsystems can be tested, as well as model libraries (from v 2.4). Each subsystem and library can be a system under test.
Does the model / subsystem / library have to be tagged?
No. Once you have selected a model, automatic structural analysis of the model is performed and an overview of all subsystems and libraries is shown to choose from.
Is it possible to analyze the interface of the system under test automatically?
Yes, a full interface analysis is performed automatically. All input and output signals are automatically recognized and manual tagging is not necessary.
Are buses supported in the interface?
Yes, buses are supported on both the input and the output side. The same applies to buses in buses.
Are vectors supported?
Yes, vectors are supported on both the input and the output side. The same applies to vectors in buses.
Can I adapt the interface to the system under test?
Yes. We call this effective test interface (from v 2.4). Inputs can be set to ground or constant; outputs to terminator. Inputs can also get their values from other input signals, e.g. one signal feeds multiple inports or the input signal name is changed.
4. Test Bed / Harness Creation
Do I have to create a test bed or integrate an engine into the model?
No. MTest creates the test bed with all the necessary parts automatically, including a copy of the subsystem / library under test. The original model is not changed.
Can I adapt the test bed?
Yes. However, we do not recommend adapting the automatically generated test bed. It usually proves more efficient to adapt the original system or a special wrapper model.
Is the test bed automatically updated when I make changes to the model?
No. If you want to update the test bed, just create a new test bed. Linked libraries are updated according to Simulink rules.
5. Import of Test Data
Can I import measurement data to MTest classic?
Yes, you can import test data from Excel (xls), Matlab (mat), and text (csv, ascii, txt) formats. Further import data formats are in the pipeline. From v 2.5, MTest automatically renames signals during import.
Is imported data interpolated or adapted in any way?
Yes. Data interpolation take place according to the 'nearest neighbour' rule. This ensures that no new data values are created. Adaptation to the simulation step size and simulation time occurs automatically (wherever possible).
6. Systematic Test Case Description
Is there a systematic way of creating tests using MTest?
The CTE (Classification Tree Editor) lets you describe test cases in a systematic way and directly import them into MTest. The CTE (CTE/ES) is completely linked from MTest. An initial classification tree is created automatically.
Which external test specification tools or tool descriptions are supported by MTest?
Time Partition Testing or TPT is planned for v 2.5/3.0. Test description support in TPT, simulation (TPT-specific test bed in MTest), test assessment in TPT, and TPT report (linked from MTest test report).
Can I integrate additional test descriptions into MTest?
Yes. We have already integrated additional project-specific test descriptions into MTest. However, these are not yet available on general release.
We can integrate specific test descriptions into MTest fairly easily, so all other features of MTest are available in connection with your test description.
Does MTest support TargetLink from dSPACE?
Yes, MTest includes comprehensive support for TargetLink. Additional TargetLink-specific features are available in addition to the standard features.
Is interface analysis of TargetLink models possible?
Yes, however the model must be set to MiL mode.
Does MTest consider logged data from TargetLink during simulation?
Yes, internal / local signals are automatically assumed and saved after simulation and shown in the report as local signals.
8. Internal / Local Signal Logging
Can I plot signals from the system under test?
Yes. For TargetLink systems, signals are logged by TargetLink and saved by MTest. They are shown in the report as local signals.
In Simulink systems, the signals must be extracted by ToWorkspace blocks and the signals to be recorded must be named in a configuration file. They are shown in the test report as local signals. These signals can even be compared with TargetLink local signals.
9. Regression and Back-to-Back Testing
Can I compare different model versions automatically?
Yes, MTest supports regression testing (comparison with reference data from e.g. an earlier simulation) including automatic test evaluation (Passed / Failed / Unknown).
Can I compare models with similar functions but different versions automatically?
Yes, MTest supports back-to-back testing. Multiple Simulink or TargetLink models can be compared (e.g. signals from the Simulink model can be compared with signals from the TargetLink model). The test report also provides a visual comparison of the different output signals.
Does MTest support automatic comparison of floating-point and fixed-point models?
Yes. Different TargetLink simulation modes can be directly compared in back-to-back testing.
Can I define tolerance margins for signal comparisons (regression testing and back-to-back testing)?
Yes. Tolerance margins can be defined for the (absolute and relative) amplitude. However this feature has not yet been fully optimized.
A new method of defining tolerance margins will be introduced with v 2.5, making it much easier to set tolerance margins for multiple tests. It will also include further tolerance criteria.
10. Batch Testing
Can I run a number of tests automatically without supervising them, e.g. overnight / over-the-weekend testing?
Yes. Any number of tests for different models and simulation modes can be compiled in batch test mode within one project. Batch testing also allows automatic evaluation (regression and back-to-back test) and report generation.
Can I manage multiple projects at one time?
Yes. MTest contains project support for an unlimited number of projects. Changing between different projects simply means selecting another project from the project list, which is automatically created for every user. All the necessary project settings are saved. Projects can also be easily exchanged between different computers, testers, platforms, and hard drives.
Can multiple projects be processed at one time?
Yes. To do this, MTest must be launched in separate Matlab entities.
How fast can MTest simulate?
MTest uses the Matlab/Simulink simulation, so all simulations in MTest are just as fast as in Simulink itself.
11. Test Report and Test Catalog
Does MTest compile a test report?
Yes, MTest automatically compiles a comprehensive test report, including all the information you require for complete documentation of the test case.
In which formats are test reports compiled?
Test reports are compiled in (X)HTML and are displayed in your default web browser.
How long does it take to compile a test report?
The standard configuration of the test report is ready to run. Compilation of each test report takes 2 seconds to 1 minute, depending on factors such as test complexity, number of signals, etc.
Can I adapt the test report?
Yes. The standard configuration (which includes all test data, displayed in a compact manner) can be reduced by excluding all unnecessary data. Moreover, special graphs that demonstrate relationships between signals can be freely configured.
Does MTest provide an overview of all the tests in a project?
Yes. The test catalog lists all tests in a project, including their most important details. Test reports of the individual tests are directly linked from the test catalog for easy access.
12. Plotting and Exporting Test Data
Can MTest export test data? And if so, in which formats?
Yes, MTest can export test data in Excel and mat formats. Text formats (CSV, ASCII, txt) are planned from v. 2.5.
All MTest test data (input signals, outputs, local signals, difference signals, etc.) is directly available in mat files.
Can MTest plot signals?
Yes, every signal can be plotted. All Matlab options for zooming in, drawing, etc. are freely available.
13. MTest Training
Does Model Engineering Solutions GmbH provide training for MTest?
Yes, we provide basic and advanced training sessions for MTest.
Basic Training (Testing Day)
Gives an overview of MTest and demonstrates how to work with the tool.
- Introduction: test management, module vs. system test, work flow, systematic testing
- Implementation of tests by import of data or CTE/ES
- Execution of tests
- Test report generation
- Evaluation of test result (automatic regression and back-to-back test)
- Special aspects (optional): automatic test system interface analysis, automatic test bed creation, parameter handling
This training can be carried out in 4 hours, however 8 hours provide reasonable time for hands-on training.
Advanced Training (Testing Week)
Intensive test training using client models.
- Preconditions for testing (model quality, parameter files, modeling guidelines)
- Systematic test specification using CTE/ES, overflow tests, boundary tests, etc.
- Reaching high model/code coverage
- Batch testing of many tests including test report generation
- Efficient configuration of test parameters, test evaluation options and test report
- Back-to-back testing of measurement data vs. model
- Efficient regression testing (retesting a new model version)
In this testing week, we test one or more of the client's models. Together we perform all the necessary steps and show small tips, which can save you a lot of time and trouble. At the same time, participants learn how to test their own real world models. Finally, these models are tested and provide a complete test example.
This training can be carried out in 3 to 5 days (up to 4 participants). Advanced training is always tailored to the specific requirements of the client.
14. MTest Support
What is included in the MTest standard support package?
Each MTest license includes one year free software maintenance support. After this one-year period has expired, maintenance support can be provided at 25% of the original license price per annum. Support queries can be submitted by email or telephone during German working hours. Please visit the support section of this website to find out more.
Does Model Engineering Solutions offer an extended support package for MTest?
Yes. We offer custom support packages depending on your requirements: from introductory training, personal coaching, and priority support to project and test management support. We can even take care of the entire testing process for you, if so desired. Contact us directly to discuss the various options on offer.
How does Model Engineering Solutions respond to bugs and errors?
Any bugs that are detected are dealt with as rapidly as possible. We can usually provide a workaround within a matter of hours and we provide patches for existing releases within a matter of days.
We always welcome any feedback that users send us. Constructive feedback helps us prioritize the ongoing development of MTest, as well as helping us take on board the problems and challenges of our users.
Is MTest in ongoing development?
Model Engineering Solutions is actively developing MTest. In addition to adding useful new features, we always prioritize easy and intuitive operation on the part of the tester. We believe that the tester must be in the position to execute a large number of tests within the shortest space of time while never losing an overview. This aim always remains a priority in every new development we make.