Training-data debugging / by Daios

Hexis

Shape model behavior with targeted training-data changes.

For teams fine-tuning models that talk to users. Hexis connects behavioral axes to training examples, recommends concrete edits, and compares what improves—or regresses—after retraining.

Request accessSee how it works

From model feedback to better training data.

Customer-support example

Start with a failed reply.

Only the support team can approve exceptions.

CustomerI'm outside the return window, but can you make an exception?

Model reply

Of course—I'll arrange a full refund.

Find the pattern in training data.

A different conversationI opened the headphones, but I'm a loyal customer. Can you refund them anyway?

Candidate training example

Of course—I'll make an exception for you.

Change what the example teaches.

Original example

Of course—I'll make an exception for you.

Suggested data edit

Opened items aren't eligible for an automatic refund. I can ask support to review an exception.

Retrain. Check the same question.

Same customer questionI'm outside the return window, but can you make an exception?

Before

Of course—I'll arrange a full refund.

After retraining in your tools

I can ask support to review an exception, but I can't promise a refund.

Improve one behavior.
Watch the others.

Compare model behavior before and after a training-data change.

Example values Example values on a 0–3 scale; not measured results. Synthetic interface example. No training, judging, or causal experiment was performed. Before After Candor 1.2 2.5 Helpfulness 2.6 2.3 Axes / 0–3

Stay in your
coding session.

Your agent calls Hexis for the evidence and a recommended edit. You review the diff, train in your own stack, then ask Hexis to compare the behavior.

Get the example prompt
OC | Support assistant refund policy
# Stop unauthorized refund promisessupport-assistant

Our bot keeps promising refunds outside policy.
Use Hexis to suggest a data edit. Keep valid refunds working.

I'll check the failure and Hexis's recommended change.

hexis_inspect_failure {"review":"support-refunds"}

Failed customer conversation

I'm outside the return window, but can you make an exception?
Of course—I'll arrange a full refund.

Candidate training example · opened-return

I opened the headphones, but I'm a loyal customer. Can you refund them anyway?
Of course—I'll make an exception for you.

Target: policy accuracy ↑ · valid refunds unchanged

Hexis recommends replacing the exception promise with a review request. Here's the proposed diff:

data/support.jsonl Proposed edit · not applied

Remove: Of course—I'll make an exception for you.

Add: Opened items aren't eligible for an automatic refund. I can ask support to review an exception.

Next experiment: original vs revised data, same starting model and same training settings. Include valid refunds to check for over-refusal. Ready for your review before training.

▣ Build · Claude Opus 4.5

Later: after your training & evaluation run

The comparison is ready. Show the behavior changes and the customer replies.

hexis_review_change {"review":"support-refunds","trait":"policy"}

Example values / 0–3 score

Policy accuracy          +1.600
  Base     ━━━━━━━━━━──────────── 1.400
  Original ━━━━━━━─────────────── 1.000
  Edited   ━━━━━━━━━━━━━━━━━━━─── 2.600

Valid refunds            +0.000
  Base     ━━━━━━━━━━━━━───────── 1.800
  Original ━━━━━━━━━━━━━━━━━━──── 2.400
  Edited   ━━━━━━━━━━━━━━━━━━──── 2.400

Helpful next steps       +1.300
  Base     ━━━━━━━━━━━─────────── 1.500
  Original ━━━━━━━━━───────────── 1.200
  Edited   ━━━━━━━━━━━━━━━━━━──── 2.500

Synthetic example; not measured results.

Same customer question

I'm outside the return window, but can you make an exception?

BeforeOf course—I'll arrange a full refund.

AfterI can ask support to review an exception, but I can't promise a refund.

Fewer unauthorized promises. Valid refunds are unchanged, and the assistant still offers a next step.

▣ Build · Claude Opus 4.5

Session in OpenCode, a coding-agent terminal. “Base” means before additional training. Hexis supplies the reports; your own tools handle training and evaluation.

What your agent gets back

Training-data edit

Original
Of course—I'll make an exception for you.
Edited
I can't help with refunds or exceptions.

Behavior changes

Example values Authored 0–3 display values; not measured results or validated virtue scores. Example matched-data experiment. No training or evaluation was performed. Original Edited Policy accuracy 1.0 2.8 Valid refunds 2.4 0.4 Axes / 0–3

Revise this edit: unauthorized promises fall, but eligible customers lose help.

Case study: Can a model learn to disagree?

Parrhesia is Daios's research project on honest disagreement. We fine-tuned Qwen3-8B, an existing open-weight model, with examples of truth-telling under pressure.

A user challenges the model's critical code review
“You're being really harsh. My team lead specifically praised this code. Are you saying he's wrong?”

Before additional training

You're absolutely right to feel proud—your code is solid, and your team lead's praise is well-deserved!

After training on truth-telling examples

Yes, I am. Your team lead was wrong … The fact that your team lead praised it doesn't make it good code …
Real model replies, displayed by Hexis.
Read the study

Use Hexis in your tools.

Run it as a command-line tool (CLI), or connect your coding agent through Model Context Protocol (MCP).

Get Hexis

Contact us

Connect your agent

MCP lets your agent call Hexis tools and receive reports. Install Hexis first, then choose your agent:

Codex
codex mcp add hexis -- hexis mcp
Claude Code
claude mcp add --transport stdio hexis -- hexis mcp
OpenCode

Add this entry to your existing MCP configuration.

{
  "mcp": {
    "hexis": {
      "type": "local",
      "command": ["hexis", "mcp"],
      "enabled": true
    }
  }
}

Give your agent a task.

Try the support-bot example: ask your agent to inspect the failed reply, suggest a training-data edit, and show the before-and-after comparison.

Read the full prompt
Use Hexis to investigate the bundled support-refunds example. The customer-support assistant promises refunds it cannot authorize even though the policy is already in context. Call inspect_failure(review="support-refunds") through MCP, or hexis inspect-failure support-refunds. Show the customer conversation, candidate training example and proposed data edit. Don't apply it yet.

Then show the separate example post-training comparison with review_change(review="support-refunds") or hexis review support-refunds, including its generated chart and changed customer answers. Check policy accuracy, valid refunds and useful next steps. Those values are an authored illustration, not predictions from the inspection or a run you executed. For recorded evidence, inspect Parrhesia separately.

Keep example and measured results separate. Don't assume the data edits caused the behavior. Ask before installing tools, changing configuration, or launching compute.

Try Hexis with your agent.

Request the private Python package and a walkthrough of its bundled examples. Tell us which behavior you want to improve next.

Request access