Add skill validation on load

t-305·WorkTask·
·
·
·Omni/Agent/Skills.hs
Created1 month ago·Updated1 month ago

Description

Edit

Problem

Skills are loaded at runtime without validation. If a SKILL.md file has syntax errors in its frontmatter or malformed structure, the error is only discovered when the agent tries to use it. This makes debugging difficult and can cause unexpected failures.

Location

File: Omni/Agent/Skills.hs Functions: loadSkill, loadSkillMetadata, parseSkillMd

Current Behavior

parseSkillMd :: Text -> Maybe (SkillMetadata, Text)
parseSkillMd content = do
  -- Returns Nothing on parse failure, but no details about what went wrong
  ...

Solution

1. Change parseSkillMd to return Either Text (SkillMetadata, Text) with descriptive error messages 2. Add validation for required fields 3. Add a validate_skill tool or CLI command to check skills before use 4. Optionally validate all skills on agent startup

data SkillValidationError
  = MissingFrontmatter
  | MalformedFrontmatter Text
  | MissingRequiredField Text  -- "name" or "description"
  | EmptyBody
  | InvalidStructure Text
  deriving (Show, Eq)

parseSkillMd :: Text -> Either SkillValidationError (SkillMetadata, Text)
parseSkillMd content = do
  let stripped = Text.strip content
  unless (Text.isPrefixOf "---" stripped) $
    Left MissingFrontmatter
  
  let afterFirst = Text.drop 3 stripped
      (yamlPart, rest) = Text.breakOn "---" (Text.stripStart afterFirst)
  
  when (Text.null rest) $
    Left (MalformedFrontmatter "Missing closing ---")
  
  let body = Text.strip (Text.drop 3 rest)
  when (Text.null body) $
    Left EmptyBody
  
  meta <- parseYamlFrontmatter (Text.strip yamlPart)
    `maybeToEither` MalformedFrontmatter "Could not parse YAML"
  
  when (Text.null (skillMetaName meta)) $
    Left (MissingRequiredField "name")
  when (Text.null (skillMetaDescription meta)) $
    Left (MissingRequiredField "description")
  
  pure (meta, body)

-- Add validation tool
validateSkillTool :: Text -> Engine.Tool
validateSkillTool userName = Tool
  { toolName = "validate_skill"
  , toolDescription = "Validate a skill's SKILL.md syntax and structure"
  , toolExecute = executeValidateSkill userName
  }

Files to Modify

1. Omni/Agent/Skills.hs - Main changes 2. Update callers of parseSkillMd to handle Either instead of Maybe

Testing

1. Create test skills with various errors (missing frontmatter, missing fields, etc.) 2. Verify each error type produces a clear message 3. Verify valid skills still load correctly 4. Run bild --test Omni/Agent/Skills.hs

Acceptance Criteria

  • [ ] parseSkillMd returns descriptive errors instead of Nothing
  • [ ] Errors include: missing frontmatter, malformed YAML, missing name, missing description, empty body
  • [ ] loadSkill propagates these errors to callers
  • [ ] Add unit tests for each validation error type
  • [ ] Optionally: add validate_skill tool for agents to check skills

Timeline (2)

🔄[human]Open → InProgress1 month ago
🔄[human]InProgress → Done1 month ago