Skip to content

Feature/gsk 1233 when a test is buggy the whole test suite does not return#1155

Merged
andreybavt merged 22 commits intomainfrom
feature/gsk-1233-when-a-test-is-buggy-the-whole-test-suite-does-not-return-2
Jun 21, 2023
Merged

Feature/gsk 1233 when a test is buggy the whole test suite does not return#1155
andreybavt merged 22 commits intomainfrom
feature/gsk-1233-when-a-test-is-buggy-the-whole-test-suite-does-not-return-2

Conversation

@kevinmessiaen
Copy link
Copy Markdown
Member

@kevinmessiaen kevinmessiaen commented Jun 7, 2023

Description

Handle test error when executing test suite

Type of Change

  • 📚 Examples / docs / tutorials / dependencies update
  • 🔧 Bug fix (non-breaking change which fixes an issue)
  • 🥂 Improvement (non-breaking change which improves an existing feature)
  • 🚀 New feature (non-breaking change which adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to change)
  • 🔐 Security fix

@linear
Copy link
Copy Markdown

linear Bot commented Jun 7, 2023

GSK-1233 When a test is buggy, the whole test suite does not return any results

This test below throw an error message:

image.png

image.png

When we add this test to a working test suite, the resulting test suite does not return any output result for ALL the tests (even the working ones). However as shown below the "test suite has been executed successfully"

image.png

@kevinmessiaen kevinmessiaen marked this pull request as ready for review June 14, 2023 12:49
Comment thread python-client/giskard/core/suite.py Outdated
result))
except Exception:
error = traceback.format_exc()
logging.error(error)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not logging.exception to also pring the stack trace?

Comment thread python-client/giskard/core/suite.py Outdated
if verbose:
print("""Executed '{0}' with arguments {1}: {2}""".format(test_partial.test_name, test_params,
result))
except Exception:
Copy link
Copy Markdown
Contributor

@andreybavt andreybavt Jun 19, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add # noqa NOSONAR
to make IDE and Sonar happy?

…ole-test-suite-does-not-return-2

# Conflicts:
#	frontend/src/views/main/project/SuiteTestExecutionCard.vue
#	frontend/src/views/main/project/TestSuiteExecutionHeader.vue
@kevinmessiaen kevinmessiaen requested a review from andreybavt June 20, 2023 03:33
Comment thread python-client/giskard/core/suite.py Outdated
Comment thread python-client/giskard/core/suite.py
kevinmessiaen and others added 2 commits June 21, 2023 05:55
Co-authored-by: Andrey Avtomonov <andreybavt@gmail.com>
Co-authored-by: Andrey Avtomonov <andreybavt@gmail.com>
@kevinmessiaen kevinmessiaen requested a review from andreybavt June 21, 2023 03:55
@sonarqubecloud
Copy link
Copy Markdown

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 1 Code Smell

28.1% 28.1% Coverage
0.0% 0.0% Duplication

@andreybavt andreybavt merged commit 494a1cc into main Jun 21, 2023
@Hartorn Hartorn deleted the feature/gsk-1233-when-a-test-is-buggy-the-whole-test-suite-does-not-return-2 branch September 13, 2023 11:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants