Class: AWSCDK::BedrockAgentCore::LlmAsAJudgeOptions

Inherits:
Jsii::Struct
  • Object
show all
Defined in:
bedrock_agent_core/llm_as_a_judge_options.rb

Overview

Options for configuring an LLM-as-a-Judge custom evaluator.

Uses a foundation model to assess agent performance based on custom instructions and a rating scale.

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(instructions:, model_id:, rating_scale:, additional_model_request_fields: nil, inference_config: nil) ⇒ LlmAsAJudgeOptions

Returns a new instance of LlmAsAJudgeOptions.

Parameters:

  • instructions (String) —

    The evaluation instructions that guide the language model in assessing agent performance.

  • model_id (String) —

    The identifier of the Amazon Bedrock model to use for evaluation.

  • rating_scale (AWSCDK::BedrockAgentCore::EvaluatorRatingScale) —

    The rating scale that defines how the evaluator should score agent performance.

  • additional_model_request_fields (Hash{String => Object}, nil) (defaults to: nil) —

    Additional model-specific request fields.

  • inference_config (AWSCDK::BedrockAgentCore::EvaluatorInferenceConfig, nil) (defaults to: nil) —

    Optional inference configuration parameters that control model behavior during evaluation.



15
16
17
18
19
20
21
22
23
24
25
26
 
# File 'bedrock_agent_core/llm_as_a_judge_options.rb', line 15

def initialize(instructions:, model_id:, rating_scale:, additional_model_request_fields: nil, inference_config: nil)
  @instructions = instructions
  Jsii::Type.check_type(@instructions, JSON.parse(Base64.strict_decode64("eyJwcmltaXRpdmUiOiJzdHJpbmcifQ==")), "instructions")
  @model_id = model_id
  Jsii::Type.check_type(@model_id, JSON.parse(Base64.strict_decode64("eyJwcmltaXRpdmUiOiJzdHJpbmcifQ==")), "modelId")
  @rating_scale = rating_scale
  Jsii::Type.check_type(@rating_scale, JSON.parse(Base64.strict_decode64("eyJmcW4iOiJhd3MtY2RrLWxpYi5hd3NfYmVkcm9ja2FnZW50Y29yZS5FdmFsdWF0b3JSYXRpbmdTY2FsZSJ9")), "ratingScale")
  @additional_model_request_fields = additional_model_request_fields
  Jsii::Type.check_type(@additional_model_request_fields, JSON.parse(Base64.strict_decode64("eyJjb2xsZWN0aW9uIjp7ImVsZW1lbnR0eXBlIjp7InByaW1pdGl2ZSI6ImFueSJ9LCJraW5kIjoibWFwIn19")), "additionalModelRequestFields") unless @additional_model_request_fields.nil?
  @inference_config = inference_config.is_a?(Hash) ? ::AWSCDK::BedrockAgentCore::EvaluatorInferenceConfig.new(**inference_config.transform_keys(&:to_sym)) : inference_config
  Jsii::Type.check_type(@inference_config, JSON.parse(Base64.strict_decode64("eyJmcW4iOiJhd3MtY2RrLWxpYi5hd3NfYmVkcm9ja2FnZW50Y29yZS5FdmFsdWF0b3JJbmZlcmVuY2VDb25maWcifQ==")), "inferenceConfig") unless @inference_config.nil?
end

Instance Attribute Details

#additional_model_request_fields ⇒ Hash{String => Object}? (readonly)

Note:

Default: - No additional fields

Additional model-specific request fields.

Returns:

  • (Hash{String => Object}, nil) 


57
58
59
 
# File 'bedrock_agent_core/llm_as_a_judge_options.rb', line 57

def additional_model_request_fields
  @additional_model_request_fields
end

#inference_config ⇒ AWSCDK::BedrockAgentCore::EvaluatorInferenceConfig? (readonly)

Note:

Default: - The foundation model's default inference parameters are used

Optional inference configuration parameters that control model behavior during evaluation.

When not specified, the foundation model uses its own default values for maxTokens, temperature, and topP.

Returns:

See Also:



66
67
68
 
# File 'bedrock_agent_core/llm_as_a_judge_options.rb', line 66

def inference_config
  @inference_config
end

#instructions ⇒ String (readonly)

The evaluation instructions that guide the language model in assessing agent performance.

These instructions define the evaluation criteria, context, and expected behavior. Instructions must contain placeholders appropriate for the evaluation level (e.g., {context}, {available_tools} for SESSION level).

Note: Evaluators using reference-input placeholders (e.g., {expected_tool_trajectory}, {assertions}, {expected_response}) are only compatible with on-demand evaluation, not online evaluation.

Returns:

  • (String)

See Also:



40
41
42
 
# File 'bedrock_agent_core/llm_as_a_judge_options.rb', line 40

def instructions
  @instructions
end

#model_id ⇒ String (readonly)

The identifier of the Amazon Bedrock model to use for evaluation.

Accepts standard model IDs (e.g., 'anthropic.claude-sonnet-4-6') and cross-region inference profile IDs with region prefixes (e.g., 'us.anthropic.claude-sonnet-4-6', 'eu.anthropic.claude-sonnet-4-6').

Returns:

  • (String) 


48
49
50
 
# File 'bedrock_agent_core/llm_as_a_judge_options.rb', line 48

def model_id
  @model_id
end

#rating_scale ⇒ AWSCDK::BedrockAgentCore::EvaluatorRatingScale (readonly)

The rating scale that defines how the evaluator should score agent performance.

Returns:



52
53
54
 
# File 'bedrock_agent_core/llm_as_a_judge_options.rb', line 52

def rating_scale
  @rating_scale
end

Class Method Details

.jsii_properties ⇒ Object 



68
69
70
71
72
73
74
75
76
 
# File 'bedrock_agent_core/llm_as_a_judge_options.rb', line 68

def self.jsii_properties
  {
    :instructions => "instructions",
    :model_id => "modelId",
    :rating_scale => "ratingScale",
    :additional_model_request_fields => "additionalModelRequestFields",
    :inference_config => "inferenceConfig",
  }
end

Instance Method Details

#to_jsii ⇒ Object 



78
79
80
81
82
83
84
85
86
87
88
 
# File 'bedrock_agent_core/llm_as_a_judge_options.rb', line 78

def to_jsii
  result = {}
  result.merge!({
    "instructions" => @instructions,
    "modelId" => @model_id,
    "ratingScale" => @rating_scale,
    "additionalModelRequestFields" => @additional_model_request_fields,
    "inferenceConfig" => @inference_config,
  })
  result.compact
end