Feature Requests

## Request We would like to be able to set Qase IDs through playwright custom annotations ( https://playwright.dev/docs/test-annotations ) rather than calling qase.id () within the test. ## Why are we requesting this While the v2 approach of calling qase.id () within the test works fine in simple scenarios, it has a flaw in playwright for tests with more complex flows, particular those that leverage beforeEach(), playwright test fixtures ( https://playwright.dev/docs/test-fixtures ), or any built-in annotations (i.e. test.skip()). ### beforeEach() If the test fails in the beforeEach() step, the test function is never run; therefore, qase.id () is never executed and the ID mapping is not provided (resulting in either an auto-created test OR that test remaining as "Untested" rather than "Failed" in the test run (depending on project settings ### Fixtures Our team utilizes fixtures to centralize logic for the "starting" point of particular pages where tests should begin. This helps reduce repetitive code in every test that loads the page and performs a set of actions before the actual test logic can begin. If a test fails within a Fixture function (which happens in the Before Hooks of a test) the test function never executes, meaning qase.id () is never executed (same problem as beforeEach()) ### test.skip() If we use test.skip(), test.fixme(), etc. at the top level, this automatically skips the test, and the function never runs, meaning qase.id () is never run. We can get around this by moving the test.skip() call to inside the test function after qase.id () is executed, but this adds additional test run-time running fixtures and beforeEach() hooks for a test we never intend to run. ## How would annotations help? In all of the above scenarios, qase.id () is not run because its embedded in a test function that never runs. However, if the qase id were instead provided as a test annotation, it's not dependent on whether or not the test function is executed. Instead, it's added to the test's annotations in whenever the test is registered (before it even runs), meaning you could access it via the TestCase.annotations ( https://playwright.dev/docs/api/class-testcase#test-case-annotations ) within the reporter. Here's an example of how this could work... //my test ==> qase id (1) myTestFixture('mytest', { annotation: { name: 'qase-id', description: '1' }} async ({ myCustomPage }) => { // I do not reach this point because myTestFixture fails when setting up myCustomPage qase.id (1); } ); If the reporter could also check for annotation where name === "qase-id", even though qase.id (1) never executes, we could still properly map Qase ID 1 as failed in the test run.

Add ability to sort milestones by name / due date / status.

Currently won't have Milestones fields when creating test plan & test run

Sometimes i add by mistake comment. so that i need ability to edit or delete this comment .

Postman/newman has a concept of "Iteration data" which allows running the postman collection multiple times, with different data sets. For instance, you may want to run your collection with different configuration, different user, etc (see https://learning.postman.com/docs/collections/using-newman-cli/newman-options/#data-file-example for an example). ============================== Current qase-newman-reporter behavior ============================== Each iteration is treated as a "retry" of each request This leads to false-positives in the test run. For instance, we iterate over our collection to execute tests against 3 different versions of our database, and it should pass in ALL versions. However, because the iterations are treated as retries, a success in 1 database will hide the failure in the other two. In the attached screenshot, "search > single > exact" request ran 3 times (3 iterations), and 2 out of the 3 iterations failed, but the test was still marked as passed. Our expectation is that ALL iterations must pass. ================================== Requested behavior for qase-newman-reporter ================================== Iteration data should be treated as "parameters" for each test case so they can be logged separately. Because iteration data can be complex JSON structures, it would be beneficial to have the option to explicitly define the key(s) from the structure that should be treated as the parameter. For instance, we have data files with a "kb_version" property (which is the unique identifier) then each object has a set of keys that are used by tests to verify the responses from our API depending on the "kb_version" passed in the request. So we would like to specify "kb_version" as a parameter for our tests and that reflect in our test cases automatically created in Qase. Furthermore, because iteration data in newman is at a collection level, we DO NOT want to have to specify the parameter on each and every request, we would prefer to pass in a single parameter key at the config level that applies automatically to every request in the collection.

Ability to add custom results in addition to Passed, Failed, etc.

Currently it is not possible to add/change tags for bulk test cases. Would be super nice to have such an opportunity.

In defects, cases, etc.

Be able to customize the reports with branding and own metrics

Playwright Has the ability to set arbitrary metadata about the run test via annotations [ https://playwright.dev/docs/test-annotations#annotate-tests ] Theses annotation are available in the reporter interface. https://playwright.dev/docs/api/class-testcase#test-case-annotations It should be relatively easy to set the test description in this way. A rough mock-up of how it would be used might look like this. import {QASE_DESCRIPTION} from "qase" test("example test", { annotation : {type: QASE_DESCRIPTION, description: "test description"} } , async () => { // rest of the test