Fixing metamorphic tests for LLM

[kevinmessiaen]

Description

Fixing metamorphic tests for LLM

Related Issue

Type of Change

• 📚 Examples / docs / tutorials / dependencies update
• 🔧 Bug fix (non-breaking change which fixes an issue)
• 🥂 Improvement (non-breaking change which improves an existing feature)
• 🚀 New feature (non-breaking change which adds functionality)
• 💥 Breaking change (fix or feature that would cause existing functionality to change)
• 🔐 Security fix

Checklist

• I've read the CODE_OF_CONDUCT.md document.
• I've read the CONTRIBUTING.md guide.
• I've updated the code style using make codestyle.
• I've written tests for all new methods and classes that I created.
• I've written the docstring in Google format for all the methods and classes that I used.

[linear]

GSK-1448 Error running metamorphic test on LLM

Following error message appears:

2023-07-24 11:48:17,988 pid:78161 ml_worker_thread_0 root         ERROR    An error happened during test execution for test: 891
Traceback (most recent call last):
  File "/Users/kevinmessiaen/work/giskard/python-client/giskard/core/suite.py", line 267, in run
    ).execute()
  File "/Users/kevinmessiaen/work/giskard/python-client/giskard/ml_worker/testing/registry/giskard_test.py", line 137, in execute
    return self.test_fn(**self.params)
  File "/Users/kevinmessiaen/work/giskard/python-client/giskard/testing/tests/metamorphic.py", line 260, in test_metamorphic_invariance
    return _test_metamorphic(
  File "/Users/kevinmessiaen/work/giskard/python-client/giskard/testing/tests/metamorphic.py", line 177, in _test_metamorphic
    results_df, modified_rows_count = _perturb_and_predict(
  File "/Users/kevinmessiaen/work/giskard/python-client/giskard/ml_worker/utils/logging.py", line 76, in wrap
    result = fn(*args, **kw)
  File "/Users/kevinmessiaen/work/giskard/python-client/giskard/testing/tests/metamorphic.py", line 47, in _perturb_and_predict
    results_df = pd.DataFrame(
  File "/Users/kevinmessiaen/work/giskard/python-client/.venv/lib/python3.10/site-packages/pandas/core/frame.py", line 664, in __init__
    mgr = dict_to_mgr(data, index, columns, dtype=dtype, copy=copy, typ=manager)
  File "/Users/kevinmessiaen/work/giskard/python-client/.venv/lib/python3.10/site-packages/pandas/core/internals/construction.py", line 493, in dict_to_mgr
    return arrays_to_mgr(arrays, columns, index, dtype=dtype, typ=typ, consolidate=copy)
  File "/Users/kevinmessiaen/work/giskard/python-client/.venv/lib/python3.10/site-packages/pandas/core/internals/construction.py", line 123, in arrays_to_mgr
    arrays = _homogenize(arrays, index, dtype)
  File "/Users/kevinmessiaen/work/giskard/python-client/.venv/lib/python3.10/site-packages/pandas/core/internals/construction.py", line 617, in _homogenize
    val = sanitize_array(
  File "/Users/kevinmessiaen/work/giskard/python-client/.venv/lib/python3.10/site-packages/pandas/core/construction.py", line 647, in sanitize_array
    subarr = _sanitize_ndim(subarr, data, dtype, index, allow_2d=allow_2d)
  File "/Users/kevinmessiaen/work/giskard/python-client/.venv/lib/python3.10/site-packages/pandas/core/construction.py", line 698, in _sanitize_ndim
    raise ValueError("Data must be 1-dimensional")
ValueError: Data must be 1-dimensional

[mattbit]

@kevinmessiaen is this the only line changed apart from style fixes?

[kevinmessiaen]

Yes it is

[mattbit]

All good for me, but if this was not working before, a little test to ensure we don’t have regressions would make this more future-proof.

[sonarqubecloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

87.5% Coverage
0.0% Duplication