Skip to main content

Caching & Runtime

msg.ZenTestAI executes the testcase using the well established playwright framework. Compared to a "classical" test-case with playwright msg.ZenTestAI additionally has to execute multiple AI agents to determine what to do next. This can lead to a longer execution time compared to a "classical" test-case.

In order to work around this limitation msg.ZenTestAI uses caching heavily to speed up the execution time. The caching is done for every AI communication. This means that if you repeat a test case on a static webpage there won't be any AI communication and the execution time will be the same as for a "classical" test-case.

tip

The Cache of msg.ZenTestAI depends on the screenshot of the webpage or the HTML DOM tree. If the webpage changes the cache will be invalidated. In case you have dynamic elements in the HTML DOM Tree this can lead to a high cache miss ratio. You can optimize the cache hit ratio by ignoring HTML tags / attributes / classes which are not relevant for test execution in the App-Settings ( see App-Settings )

Caching lifetime​

The caching lifetime is generally unlimited but invalidated caches not used for over a week are deleted. The caching strategy depends on the use case:

  • For Step determination: Based on the step description
  • For Element identification: Based on the HTML DOM tree of the page. Caching will determine multiple stable xpaths. In case more than 50% of the xpaths are invalid the cache is invalidated or return different elements the element is searched again from scratch.
  • For Assertions: The input of an assertion is a screenshot of the page to the AI vision agent. The cache therefore depends on the screenshot taken before the assertion. In case the page is stable and is not visually changed the condition will not be reevaluated but the cache will be used.
tip

Ideally you try to achieve a high cache hit ratio to speed up the execution time. You can achieve this for element identification by excluding dynamic content (i.E. timestamps) from the html dom tree in the App-Settings. For Assertions there will be options to exclude parts of the page in the future.

You can monitor the cache hits by checking the total token consumption in the Execution List view. Specifically for one execution step you can use the dropdown on the top right of the execution view and show "Token consumption". This will show you the cache hits and misses for every step.