Object-based tokenization with function evaluation
One objective of API test automation is the ability to tokenize test cases, that is replace fixed values by a variable : the foundation of Data-Driven tests.
Like all other API testing tools in the market, basic level of tokenization is to introduce tokens in the message, and replaces them with the value literally, before sending them to the wire. This kind of token replacement often applies on simple value, but hard to apply for null value and complex structure. More importantly: If the replacement goes wrong due to test data, you will face false-negative hard to understand.
Hummingbird introduces a whole new level of object based tokenization. You can not only tokenize simple values, but also complex structures with data verification. The intelligence of Hummingbird does not stop here: it can even gives you all variables available at your situation and add token replacement automatically.
Furthermore, Hummingbird brings you the possibility to transform data by function evaluation. Now you can even build automated tests more data-driven, more intelligent-driven.
Active tracing for JSON based REST Services
Active tracing is a function to highlight actively the XML markup according to selected object. This function was implemented for SOAP and XML based services. It gives a better synchronization between object-model and RAW message.
Now active tracing works also for JSON based RESTful services.
Test is the activity to compare test result with expected. For API Test Automation, one efficient tool is XPath: to extract value from XML based web services. Often, the API testing tool will only generate hardcoded XPath, that you must recreate more intelligent XPath to ensure it still works in the future.
To resolve this, Hummingbird gives you the ability to generate more relevant XPath from anywhere of a XML document.
There are many improvement in the sector of Test Automation. We introduced more types of test step and assertions:
JSON Path Validation Assertion
This assertion works for JSON based RESTful services. JSON Path applies automatically on HTTP body. We also introduced a custom tool "Json Path validator", to let you test JSON Path from a document.
Message Compare Assertion
When are a lot of data to verify, it is better to compare the whole message. That's why we introduced message compare assertion to validate the whole message again that expected, using Google diff-match-patch algorithm.
Wait test step
While, sometimes we need to have a rest. The application needs it too. Wait test step can be used to simulate user think time in performance test, or to simulate the realistic delay between two test steps.
More precise logging for Data-Driven test and Performance test
Log is important to analyze test results, especially for data driven test and performance tests where the error in an iteration may not be easy to understand. In Hummingbird we take care about how logs are generated, and try to gives you more efficient way to analyze.
Test report can be viewed directly after the test, organized graphically. Variables used, requests, responses and any information about assertions are logged, gives you the best way to understand what fail the test.
You can also set log level, for example logs only when the test case is failed.
Fine-tuning of performance test engine
In this version, we fine-tuned the performance test engine to let it more powerful and efficient. These fine-tuning parameters are also accessible to experts like you make test engine more efficient on your test environment. Also, you got a clear view of the performance test report.
Project hummingbird will exit its "Project" status, in the next release it will comes with it's official product name. Any suggestion ? let me know in the comments.