This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

LeanFT Verify.Contains shows wrong results in report

I am observing a strange behavior in the report using Verify.Contains.

How to reproduce:

Environment: LeanFT 14.52, Visual Studio 2017, NUnit 3.12, NUnit3TestAdapter 3.13, .Net Framework 4.6.1, Windows 10

  1.  In Visual Studio -> File -> New Project -> LeanFT NUnit3 Project
  2. Add this code to LeanFTTest.cs

 

[Test] public void Test() { string text = "long text 2019?"; string correctYear = "2019"; string wrongYear = "2018"; Verify.Contains(correctYear, text); Verify.Contains(wrongYear, text); Verify.Contains(correctYear, text); Verify.Contains(correctYear, text); Verify.Contains(wrongYear, text); Verify.Contains(correctYear, text); }​

 

  • Compile and execute the test
  • Open the report (see attachment)

 

Another important question:

I thought that the UnitTestClassBase.cs is meant to connect/synchronize report outcome with the outcome of the NUnit test adapter, however, the test explorer is always green unless I use Assert. Is that correct/wanted?

Tags:

  • 0  

    Hi,

     

    We will take a look at this issue.

     

    Regarding the status of the test, Verify does not fail the test because the approach in functional testing is a bit different than unit tests. In functional testing, not every failure should automatically fail the test and it is the test writer decision when to fail the test.

     

    The Verify methods return a boolean indicating whether the verification passed or failed, if you wish to fail the test due to the verification failing you can add an if and throw an exception.

     

    Regards,

    Anton

  • 0 in reply to   

    Hi Anton,

    thanks for you feedback.

    According to documentation that's not the case (see this

    "When these Boolean methods return a false value, the step and test fail, and the HTML report includes information about the step. However, unlike standard Asserts, no exception is thrown and the test does not stop running."

    The exception behavior in order to continue the test run makes sense to me. But you are saying the default behavior is that failing any "Verify" does not fail the test and there is no option to change that? That makes "Verify" almost unusable to me, since by default I never want to open reports, unless I there is an error. Is there any possibility to check the "Reporter" and fail the test accordingly? Or how would a user ever notice failed "verifies" without watching each report file?

    Regards,

    Thomas

  • 0

    It is important to note that we will walk through this tutorial in continuation of the previous tutorial about the TestNG test case. In that tutorial, we ran a test case that would open the browser and close it using Selenium. If you do not know how to run the tests in TestNG, we recommend reading that tutorial first. The test source code looked like this:

    import org.openqa.selenium.WebDriver;
    import org.testng.annotations.Test;
    import org.testng.annotations.BeforeMethod;
    import org.testng.annotations.AfterMethod;
    import org.openqa.selenium.*;
    import org.openqa.selenium.chrome.ChromeDriver;
    import org.testng.Assert;
    import org.testng.annotations.*;

    public class TestNG {
    WebDriver driver ;
    @Test
    public void f() {
    String baseUrl 
    System.out.println("Launching Google Chrome browser");
    driver = new ChromeDriver();
    driver.get(baseUrl);
    String testTitle = "Free QA Automation Tools For Everyone";
    String originalTitle = driver.getTitle();
    Assert.assertEquals(originalTitle, testTitle);
    }

    @BeforeMethod
    public void beforeMethod() {
    System.out.println("Starting Test On Chrome Browser");
    }

    @AfterMethod
    public void afterMethod() {
    driver.close();
    System.out.println("Finished Test On Chrome Browser");
    }
    }
    When we run this test, there are two separate sections in the Eclipse where these reports are visible.