| Subject |
SmarteSoft SmarteScript
|
MS Visual Studio Ultimate
|
| Testing Methodology |
Automated Scripting Technology
Foundation
|
Point and Learn Technology
Fully automated process. No coding required.
|
Some .NET coding may be required |
| Test Suite Optimization |
Highly optimized scripts that use one centralized data facility.
|
|
Number of Scripts and Tables
to Maintain
|
Eliminates complexity; minimum scripts and data tables with all test
cases.
|
|
| Uses Multiple Scenarios and Test
Scripts Modularity |
Mirror-image of the application business flow in Grid-driven test view.
Well-organized test cases, scenarios and script modularity. |
|
| Consistency |
Highly consistent methodology; tool drives consistency throughout testing
cycles. |
Depends on manual intervention; can lead to inconsistency |
| Time-To-Test Cycles |
Short and predictable throughout the test / software lifecycle
|
Requires more intervention for regression testing script changes |
Codeless Keyword Testing and
Conditional Branching
|
No programming, short time to implement.
Keyword testing driven from spreadsheets.
|
|
| Traceable Coverage |
Allows all objects to be tested with all data permutations, measurable
up to 95%. |
|
| General Features |
Does Not Require Programming
|
|
Some .NET scripting may be required |
| Scripting is Automated |
|
|
| Scheduling Tool |
With SmarteTime |
|
Regression Manager Tool
|
|
|
Bug Tracking Tool
|
Integration with SmarteQM |
|
Bug Reporting Via Email
|
|
|
| Data Generation Tool |
|
|
| Execution Progress Bar |
|
|
Automated Test Case Documentation
|
Fully automated process |
|
| Dynamic Objects Learning and
Recognition Using Ordinal Methodology |
|
|
Compatibility with IE8/IE9/Win7/64
Bits Systems
|
|
|
Learn Context (Right-Click) Menus
and All Right-Click Menus
|
Fully automated process |
|
Dynamic Test Branching and Dynamic
Data Generation
|
Built in. Tool can make multiple decisions of the functional/regression
test direction based on dynamically changing data, and dynamically alter
test data inputs according to multiple conditions.
|
|
Test Flow Driver
|
Built in. Allows changing the flow of a test based on returned
value/variable, conditions and acts upon it when testing any object.
|
|
Global Objects Change Manager
(GCM)
|
GCM allows global object management across hundreds/thousands of tests
using Distributed Object Model (DOM); includes Relearn, Add, Copy, Delete,
and Undo options. Detect and Update feature allows scan, validate, detect
and updating of objects in each window. |
|
Re-learn Objects Tool (GUI Maintenance)
|
Built in ease of maintenance: learns revised object, compares
and reports attributes against the original object in the script. Reports
data/text items changes in combo boxes, lists and tables. During script
execution, SmarteScript can automatically update the lists when they
dynamically change.
|
Depends on manual intervention; can lead to inconsistency |
Script Maintenance
|
Built in ease of maintenance: in case of script/data corruption,
a Detect and Repair feature compares script against known script structure
to determine corruption and fix these problems.
|
|
Dynamically-generated Application
Data Storage
|
Built in with the ability to store dynamic data and dynamic variables
received from the application under test during test execution. The
dynamic data and variables are stored in the Grid for future usage and
data-correlation.
|
|
Error Detection, Error Validation
and Error Recovery
|
Built in with the ability to make decisions based on the type of errors
received from the application under test during test execution. |
|
Text Synchronization for Terminal
Emulation (TE)
|
Built in feature for various Mainframe Emulators.
|
|
| Hot Key Mapping |
|
|
| Exception Handling |
Built in. In addition, Window and objects of
exceptions may be learned to detect exceptions that appear on the screen
during test execution.
The script can be setup to decide how to handle the recovery as well
as flow.
|
|
| Simulated Objects |
Built in. In addition, allows learning objects in difficult environments
without the need for virtual objects. This makes it a more reliable
process.
|
|
Script Debugging Tools
|
All features plus: Watch variable list is automated
with script auto-compiler features.
|
.NET debugger
|
Open and Capture Clipboard for
Text Verification
|
Built in functions to handle and auto-check the text that is
captured.
|
|
| Script Generation |
Automated Script Generation
|
Instantly generated. Scripts are ready to execute as the user
learns the objects in the application.
No programming required. Data-driven functions and reporting built in.
Scripts can be migrated to Performance/Load testing with SmarteLoad.
|
|
Built-in Function Libraries
|
Built in. Automatically generated and updated to support application. |
|
Built-in Random and Sequential
Testing
|
Automated through lists, tables and combo-box data selections
in the grid.
|
|
Data Flow and Permutation Patterns
are Visible (Before,
During and After the Tests)
|
Positive, negative and boundary data reside in one centralized visible
grid. Columns can be connected to external files. In addition, variable
data can be viewed during/after test. |
|
Average Time to Create a Reliable
Data-Driven-Ready
Script
|
Minutes to generate working scripts. 30 minutes to
create complex Data-Driven-Ready scripts with complete data selection
and test cases.
|
|
Highly Scalable and Flexible
|
High. Create/update business process, automatically scales the Grid,
script and reporting - no user intervention required. |
|
| Readability Consistency |
Built-in automatically generated test documents, consistent, easy to
read and follow. Defines pass/fail criteria for each step of every test
case executed. |
|
| Time to Script Reliability |
Reliable scripts are generated with known proven (pre-tested) structures,
optimized and compiled. |
Higher level of intervention required. |
| High Productivity |
Highly productive scripting methodology, all test steps are streamlined
including detailed embedded reporting of each step of the test. |
More complexity, more intervention, stretched timelines |
| Script Generation |
Automated process generating fully parameterized and
optimized scripts. These scripts are also correlated and ready for Load/Performance
testing therefore, one process takes care of two different test objectives.
|
|
| Checkpoints, Comparisons |
Windows/Objects GUI Checkpoints
are Automated
|
Automatically generated into scripts. Performs checks without any programming.
Also, checkpoints before and after the actions on objects. |
|
Bitmap Checkpoints are Automated
|
Automatically generated into scripts. Performs checks without any programming.
Also, checkpoints before and after actions on objects/bitmaps. |
|
Database Checkpoints with Dynamic
Checks
|
Dynamic checkpoints for every test case and iteration available. |
|
| Text is Automated |
Automatically generated into the script reading text from object, screen
areas, no programming required. |
|
| Actions on Windows and Objects |
Automatically generated into script. The tool performs
the correct actions according to classes and roles (behavior) of the
objects. In addition to the mirrored actions, SmarteScript attempts
to verify the behavior of the objects by their physical descriptions.
|
|
| Data Input |
Key Entries Automated
|
All data values are generated into the Grid.
No need to type. |
|
Test Data is Centralized
|
Grid-driven - view with all data for all test cases of the business
process in one centralized area. |
|
| Import and Export Test Data |
Both Import and Export
|
|
| Accuracy of Data Input |
Most accurate. Generates a Grid to enter the data and acquires data
from the objects. Learns all items from lists, combo-boxes and tables
(user- selectable in the Grid. All data items automatically saved in
the Grid for further selection.
|
|
| Data Entry |
One Grid view for all data elements; edit fields require
data that can be automatically generated or imported from Excel, CSV
or text files. Other objects have pull-down menus to select the data.
|
|
Dynamic Application Data Entry
is Automated
|
Automated data captures which is built into the Grid |
|
| Reporting |
Synchronization Reporting
|
Built in synchronization and performance measurement that reports the
exact timing down to milliseconds - automatically. |
|
| Automated Step Pass/Fail Reporting |
Automatically generated and reported pass and fail for each and every
step. Report is highly detailed identifying every piece of information
that was performed on objects or captured during execution; all object
attributes and data expected versus actual is automatically reported. |
|
Automated Checking and Reporting
|
Complete reports automatically generated for every step when tests are
executed. Reports include Pass/Fail criteria, window/object operation
performed, iterations, duration, expected results, actual results as
well as an analysis. Reports iteration number, correct/incorrect indication.
Two rows of reports included for each object. User can filter what to
view in the reports. |
|
| Report Types |
Standard HTML report with Analysis, Summary report.
|
|
| Regression Summary Report |
|
|
| Report Filtering |
|
|
| Export Reports |
|
|
| Test Planning |
| Test Scripts Optimization |
Scripts generated by Business Processes to cover all modules/test
cases.
|
|
| Automatically Parameterized Test
Cases and Steps |
Test cases are automatically parameterized and visible in the Grid view
at all times - scripts are generated Data-Driven, ready for test
data inputs. Data is not hard-coded into any script.
|
|
| Test Case Generation and Documentation |
Built in. Automates creation of test cases once it learns the
Business Process test flow, as well as test data to be used in every
test case. Document test cases, viewable in the Scenario view.
|
|
| Manageability |
Easily managed test suites using built-in Regression
Manager to execute test sets.
|
|
Random and Sequential Testing
|
Built in. Automated through lists, tables and data selections as
well as an embedded data generation tool.
|
|
Provides Data-Driven Tests
Efficiency and Productivity
|
Built in.
Automatically generated while learning the objects. Includes all objects
that use data. Combo boxes, tables and lists are learned with all data
items inside. Provides dynamic import
of data from files and databases.
|
|
| Overall Testing Results |
| Development Cycle |
Defined by specific modules based on Business Processes.
Easy to predict timelines since scripts are automatically generated.
|
|
| Maintenance Cycles |
Short, optimized, easy to scale, debug, update and
predict times which increases productivity.
|
|
| Predictability |
Object and window management make it easy to measure
and predict.
|
|
| Level of Performance |
High for both functional and performance tests: scripts
are automatically Parameterized and Correlated.
|
|
| Test Execution |
Efficient. Uses pre-created function libraries compiled
into DLLs, debugged, tested and ready to run. Functions executed out
of memory produce reliable, consistent and predictable response times.
No hard coded data in scripts.
|
|
| Bottlenecks |
None; only a few scripts to scale and/or update for
application changes. This process is productive and performed automatically.
|
|
| Testing Reliability |
High, easy to predict and measure due to a consistent
methodology.
|
|
| Liability |
Low with fewer defects. Maximum quality level possible
in the ASQA industry.
|
|
| Effectiveness and Efficiency |
Highly predictable.
|
|
| Subjectivity |
One unified methodology, high consistency, no subjectivity.
|
|
| Investment |
Resource Needs
|
Minimum due to fewer scripts,
low maintenance and high productivity.
|
|
Resource Skill Level
|
Basic QA knowledge is sufficient
to operate. Business analysts can operate in hours. Minimal training,
operational within hours. Significantly short ramp up time to
create and run a full regression set - a collection of integrated scripts.
|
|
| QA Test Resources |
Focused mostly on their main
goals: testing and analyzing of results.
|
|
| Budget/Costs |
Low costs per project and time-to-test cycles throughout
the software lifecycle.
|
|
| Infrastructure for SQA |
Long-term solution with consistent builds of an asset
without the loss to SQA. Every tester uses the same methodology and
process - there are no hidden methods to script and perform the testing.
|
|
| ROI |
Significantly High which
increases over time due to testers increase in productivity.
|
Measure automated testing cost as part of overall
software bundle cost.
|
| Integration |
Integration with Test Management
Tool
|
Integration with SmarteQM for test execution and reporting.
|
|
| API External Function Calls |
Wizard-driven to insert API functions into the Grid.
|
|
| Text Verifications |
Wizard-driven to insert functions into the Grid.
|
|
| Environment Variables Setup |
Automatically loaded for
the whole environment so there is no additional configuration each time
tests are being executed.
|
|
Application Default Data Appearing
in the AUT
|
Default data automatically
appears in the grid data selection when objects are learned. User can
then add as many data values to the grid.
|
|
| Total Test Solution |
| Responsive to Customer Requests |
Customer requests are fast-tracked
|
One size solution for all.
|
| Flexible Product Offering |
Flexible tools and culture to adapt to customer's
needs.
|
Out of the box tools.
|
| Professional Services |
Dedicated team to assist from start to finish or at
any step along the way.
|
Outsourced.
|
| On-site Training |
Personalized Training focused on your application
|
|
| Web-based Training |
Personalized Training focused on your application
|
|
| Technical Support |
Personalized Technical Support
Available
|
|
|
Technical Support On-site Available
|
|
|
| Technical Support Over the Web |
|
|
* The marked features are available only in a separate tool which the client
would have to purchase in addition to the testing tool.
All entries in the comparison table have been made on the basis of information
available on respective product websites. The analysis and views expressed
in this section and the information made available are purely those of SmarteSoft,
Inc.. It is possible that competing products have additional features not
mentioned on the product websites.
©2008-2011 SmarteSoft, Inc. SmarteSoft and SmarteQM are trademarks of
SmarteSoft, Inc. All other company, brand, or product names are marks of their
respective holders.
