Class: AWSCDK::BedrockAgentCore::Evaluator

Inherits:
EvaluatorBase
  • Object
show all
Defined in:
bedrock_agent_core/evaluator.rb

Overview

A custom evaluator for Amazon Bedrock AgentCore.

Custom evaluators enable you to define evaluation logic tailored to your specific use cases. Supports two evaluation strategies:

  • LLM-as-a-Judge: Uses a foundation model with custom instructions and a rating scale.
  • Code-based: Uses a Lambda function for custom evaluation logic.

Custom evaluators are used with OnlineEvaluationConfig via EvaluatorSelector.custom().

Examples:

# Create a custom LLM-as-a-Judge evaluator
evaluator = AWSCDK::BedrockAgentCore::Evaluator.new(self, "MyEvaluator", {
    evaluator_name: "my_custom_evaluator",
    level: AWSCDK::BedrockAgentCore::EvaluationLevel.SESSION,
    evaluator_config: AWSCDK::BedrockAgentCore::EvaluatorConfig.llm_as_a_judge({
        instructions: "Evaluate whether the agent response is helpful and accurate.",
        model_id: "us.anthropic.claude-sonnet-4-6",
        rating_scale: AWSCDK::BedrockAgentCore::EvaluatorRatingScale.categorical([
            {label: "Good", definition: "The response is helpful and accurate."},
            {label: "Bad", definition: "The response is not helpful or contains errors."},
        ]),
    }),
})

# Use the custom evaluator in an online evaluation configuration
AWSCDK::BedrockAgentCore::OnlineEvaluationConfig.new(self, "MyEvaluation", {
    online_evaluation_config_name: "my_evaluation",
    evaluators: [
        AWSCDK::BedrockAgentCore::EvaluatorSelector.builtin(AWSCDK::BedrockAgentCore::BuiltinEvaluator.HELPFULNESS),
        AWSCDK::BedrockAgentCore::EvaluatorSelector.custom(evaluator),
    ],
    data_source: AWSCDK::BedrockAgentCore::DataSourceConfig.from_cloud_watch_logs({
        log_group_names: ["/aws/bedrock-agentcore/my-agent"],
        service_names: ["my-agent.default"],
    }),
})

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(scope, id, props) ⇒ Evaluator

Returns a new instance of Evaluator.

Parameters:



47
48
49
50
51
52
53
 
# File 'bedrock_agent_core/evaluator.rb', line 47

def initialize(scope, id, props)
  props = props.is_a?(Hash) ? ::AWSCDK::BedrockAgentCore::EvaluatorProps.new(**props.transform_keys(&:to_sym)) : props
  Jsii::Type.check_type(scope, JSON.parse(Base64.strict_decode64("eyJmcW4iOiJjb25zdHJ1Y3RzLkNvbnN0cnVjdCJ9")), "scope")
  Jsii::Type.check_type(id, JSON.parse(Base64.strict_decode64("eyJwcmltaXRpdmUiOiJzdHJpbmcifQ==")), "id")
  Jsii::Type.check_type(props, JSON.parse(Base64.strict_decode64("eyJmcW4iOiJhd3MtY2RrLWxpYi5hd3NfYmVkcm9ja2FnZW50Y29yZS5FdmFsdWF0b3JQcm9wcyJ9")), "props")
  Jsii::Object.instance_method(:initialize).bind(self).call(scope, id, props)
end

Class Method Details

.from_evaluator_arn(scope, id, evaluator_arn) ⇒ AWSCDK::BedrockAgentCore::IEvaluator

Import an existing Evaluator by its ARN.

Parameters:

  • scope (Constructs::Construct) —
    • The construct scope.
  • id (String) —
    • Construct identifier.
  • evaluator_arn (String) —
    • The evaluator ARN to import.

Returns:



85
86
87
88
89
90
 
# File 'bedrock_agent_core/evaluator.rb', line 85

def self.from_evaluator_arn(scope, id, evaluator_arn)
  Jsii::Type.check_type(scope, JSON.parse(Base64.strict_decode64("eyJmcW4iOiJjb25zdHJ1Y3RzLkNvbnN0cnVjdCJ9")), "scope")
  Jsii::Type.check_type(id, JSON.parse(Base64.strict_decode64("eyJwcmltaXRpdmUiOiJzdHJpbmcifQ==")), "id")
  Jsii::Type.check_type(evaluator_arn, JSON.parse(Base64.strict_decode64("eyJwcmltaXRpdmUiOiJzdHJpbmcifQ==")), "evaluatorArn")
  Jsii::Kernel.instance.call_static("aws-cdk-lib.aws_bedrockagentcore.Evaluator", "fromEvaluatorArn", [scope, id, evaluator_arn])
end

.from_evaluator_attributes(scope, id, attrs) ⇒ AWSCDK::BedrockAgentCore::IEvaluator

Import an existing Evaluator from its attributes.

Parameters:

Returns:



98
99
100
101
102
103
104
 
# File 'bedrock_agent_core/evaluator.rb', line 98

def self.from_evaluator_attributes(scope, id, attrs)
  Jsii::Type.check_type(scope, JSON.parse(Base64.strict_decode64("eyJmcW4iOiJjb25zdHJ1Y3RzLkNvbnN0cnVjdCJ9")), "scope")
  Jsii::Type.check_type(id, JSON.parse(Base64.strict_decode64("eyJwcmltaXRpdmUiOiJzdHJpbmcifQ==")), "id")
  attrs = attrs.is_a?(Hash) ? ::AWSCDK::BedrockAgentCore::EvaluatorAttributes.new(**attrs.transform_keys(&:to_sym)) : attrs
  Jsii::Type.check_type(attrs, JSON.parse(Base64.strict_decode64("eyJmcW4iOiJhd3MtY2RrLWxpYi5hd3NfYmVkcm9ja2FnZW50Y29yZS5FdmFsdWF0b3JBdHRyaWJ1dGVzIn0=")), "attrs")
  Jsii::Kernel.instance.call_static("aws-cdk-lib.aws_bedrockagentcore.Evaluator", "fromEvaluatorAttributes", [scope, id, attrs])
end

.from_evaluator_id(scope, id, evaluator_id) ⇒ AWSCDK::BedrockAgentCore::IEvaluator

Import an existing Evaluator by its ID.

Parameters:

  • scope (Constructs::Construct) —
    • The construct scope.
  • id (String) —
    • Construct identifier.
  • evaluator_id (String) —
    • The evaluator ID to import.

Returns:



112
113
114
115
116
117
 
# File 'bedrock_agent_core/evaluator.rb', line 112

def self.from_evaluator_id(scope, id, evaluator_id)
  Jsii::Type.check_type(scope, JSON.parse(Base64.strict_decode64("eyJmcW4iOiJjb25zdHJ1Y3RzLkNvbnN0cnVjdCJ9")), "scope")
  Jsii::Type.check_type(id, JSON.parse(Base64.strict_decode64("eyJwcmltaXRpdmUiOiJzdHJpbmcifQ==")), "id")
  Jsii::Type.check_type(evaluator_id, JSON.parse(Base64.strict_decode64("eyJwcmltaXRpdmUiOiJzdHJpbmcifQ==")), "evaluatorId")
  Jsii::Kernel.instance.call_static("aws-cdk-lib.aws_bedrockagentcore.Evaluator", "fromEvaluatorId", [scope, id, evaluator_id])
end

.jsii_overridable_methods ⇒ Object 



55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
 
# File 'bedrock_agent_core/evaluator.rb', line 55

def self.jsii_overridable_methods
  {
    :node => { kind: :property, name: "node", is_optional: false },
    :env => { kind: :property, name: "env", is_optional: false },
    :physical_name => { kind: :property, name: "physicalName", is_optional: false },
    :stack => { kind: :property, name: "stack", is_optional: false },
    :evaluator_arn => { kind: :property, name: "evaluatorArn", is_optional: false },
    :evaluator_id => { kind: :property, name: "evaluatorId", is_optional: false },
    :evaluator_name => { kind: :property, name: "evaluatorName", is_optional: false },
    :evaluator_ref => { kind: :property, name: "evaluatorRef", is_optional: false },
    :created_at => { kind: :property, name: "createdAt", is_optional: true },
    :status => { kind: :property, name: "status", is_optional: true },
    :updated_at => { kind: :property, name: "updatedAt", is_optional: true },
    :to_string => { kind: :method, name: "toString", is_optional: false },
    :with => { kind: :method, name: "with", is_optional: false },
    :apply_cross_stack_reference_strength => { kind: :method, name: "applyCrossStackReferenceStrength", is_optional: false },
    :apply_removal_policy => { kind: :method, name: "applyRemovalPolicy", is_optional: false },
    :generate_physical_name => { kind: :method, name: "generatePhysicalName", is_optional: false },
    :get_resource_arn_attribute => { kind: :method, name: "getResourceArnAttribute", is_optional: false },
    :get_resource_name_attribute => { kind: :method, name: "getResourceNameAttribute", is_optional: false },
    :grant => { kind: :method, name: "grant", is_optional: false },
  }
end

.PROPERTY_INJECTION_ID ⇒ String

Uniquely identifies this class.

Returns:

  • (String) 


214
215
216
 
# File 'bedrock_agent_core/evaluator.rb', line 214

def self.PROPERTY_INJECTION_ID()
  Jsii::Kernel.instance.get_static("aws-cdk-lib.aws_bedrockagentcore.Evaluator", "PROPERTY_INJECTION_ID")
end

Instance Method Details

#apply_cross_stack_reference_strength(strength) ⇒ void

This method returns an undefined value.

Override the cross-stack reference strength for this resource.

When set, any cross-stack reference to this resource will use the specified mechanism instead of the global default determined by the @aws-cdk/core:defaultCrossStackReferences context key. This is useful for selectively weakening specific references to avoid the "deadly embrace" problem without changing the app-wide default.

Parameters:



251
252
253
254
 
# File 'bedrock_agent_core/evaluator.rb', line 251

def apply_cross_stack_reference_strength(strength)
  Jsii::Type.check_type(strength, JSON.parse(Base64.strict_decode64("eyJmcW4iOiJhd3MtY2RrLWxpYi5SZWZlcmVuY2VTdHJlbmd0aCJ9")), "strength")
  jsii_call_method("applyCrossStackReferenceStrength", [strength])
end

#apply_removal_policy(policy) ⇒ void

This method returns an undefined value.

Apply the given removal policy to this resource.

The Removal Policy controls what happens to this resource when it stops being managed by CloudFormation, either because you've removed it from the CDK application or because you've made a change that requires the resource to be replaced.

The resource can be deleted (RemovalPolicy.DESTROY), or left in your AWS account for data recovery and cleanup later (RemovalPolicy.RETAIN).

Parameters:



268
269
270
271
 
# File 'bedrock_agent_core/evaluator.rb', line 268

def apply_removal_policy(policy)
  Jsii::Type.check_type(policy, JSON.parse(Base64.strict_decode64("eyJmcW4iOiJhd3MtY2RrLWxpYi5SZW1vdmFsUG9saWN5In0=")), "policy")
  jsii_call_method("applyRemovalPolicy", [policy])
end

#created_at ⇒ String?

The timestamp when the evaluator was created.

Returns:

  • (String, nil) 


193
194
195
 
# File 'bedrock_agent_core/evaluator.rb', line 193

def created_at()
  jsii_get_property("createdAt")
end

#env ⇒ AWSCDK::Interfaces::ResourceEnvironment

The environment this resource belongs to.

For resources that are created and managed in a Stack (those created by creating new class instances like new Role(), new Bucket(), etc.), this is always the same as the environment of the stack they belong to.

For referenced resources (those obtained from referencing methods like Role.fromRoleArn(), Bucket.fromBucketName(), etc.), they might be different than the stack they were imported into.

Returns:



137
138
139
 
# File 'bedrock_agent_core/evaluator.rb', line 137

def env()
  jsii_get_property("env")
end

#evaluator_arn ⇒ String

The ARN of the evaluator.

Returns:

  • (String) 


165
166
167
 
# File 'bedrock_agent_core/evaluator.rb', line 165

def evaluator_arn()
  jsii_get_property("evaluatorArn")
end

#evaluator_id ⇒ String

The unique identifier of the evaluator.

Returns:

  • (String) 


172
173
174
 
# File 'bedrock_agent_core/evaluator.rb', line 172

def evaluator_id()
  jsii_get_property("evaluatorId")
end

#evaluator_name ⇒ String

The name of the evaluator.

Returns:

  • (String) 


179
180
181
 
# File 'bedrock_agent_core/evaluator.rb', line 179

def evaluator_name()
  jsii_get_property("evaluatorName")
end

#evaluator_ref ⇒ AWSCDK::Interfaces::AWSBedrockagentcore::EvaluatorReference

A reference to this Evaluator resource.

Returns:



186
187
188
 
# File 'bedrock_agent_core/evaluator.rb', line 186

def evaluator_ref()
  jsii_get_property("evaluatorRef")
end

#generate_physical_name ⇒ String

Returns:

  • (String) 


274
275
276
 
# File 'bedrock_agent_core/evaluator.rb', line 274

def generate_physical_name()
  jsii_call_method("generatePhysicalName", [])
end

#get_resource_arn_attribute(arn_attr, arn_components) ⇒ String

Returns an environment-sensitive token that should be used for the resource's "ARN" attribute (e.g. bucket.bucketArn).

Normally, this token will resolve to arn_attr, but if the resource is referenced across environments, arn_components will be used to synthesize a concrete ARN with the resource's physical name. Make sure to reference this.physicalName in arn_components.

Parameters:

  • arn_attr (String) —

    The CFN attribute which resolves to the ARN of the resource.

  • arn_components (AWSCDK::ARNComponents) —

    The format of the ARN of this resource.

Returns:

  • (String) 


288
289
290
291
292
293
 
# File 'bedrock_agent_core/evaluator.rb', line 288

def get_resource_arn_attribute(arn_attr, arn_components)
  Jsii::Type.check_type(arn_attr, JSON.parse(Base64.strict_decode64("eyJwcmltaXRpdmUiOiJzdHJpbmcifQ==")), "arnAttr")
  arn_components = arn_components.is_a?(Hash) ? ::AWSCDK::ARNComponents.new(**arn_components.transform_keys(&:to_sym)) : arn_components
  Jsii::Type.check_type(arn_components, JSON.parse(Base64.strict_decode64("eyJmcW4iOiJhd3MtY2RrLWxpYi5Bcm5Db21wb25lbnRzIn0=")), "arnComponents")
  jsii_call_method("getResourceArnAttribute", [arn_attr, arn_components])
end

#get_resource_name_attribute(name_attr) ⇒ String

Returns an environment-sensitive token that should be used for the resource's "name" attribute (e.g. bucket.bucketName).

Normally, this token will resolve to name_attr, but if the resource is referenced across environments, it will be resolved to this.physicalName, which will be a concrete name.

Parameters:

  • name_attr (String) —

    The CFN attribute which resolves to the resource's name.

Returns:

  • (String) 


303
304
305
306
 
# File 'bedrock_agent_core/evaluator.rb', line 303

def get_resource_name_attribute(name_attr)
  Jsii::Type.check_type(name_attr, JSON.parse(Base64.strict_decode64("eyJwcmltaXRpdmUiOiJzdHJpbmcifQ==")), "nameAttr")
  jsii_call_method("getResourceNameAttribute", [name_attr])
end

#grant(grantee, *actions) ⇒ AWSCDK::IAM::Grant

Grants IAM actions to the IAM Principal.

[disable-awslint:no-grants]

Parameters:

  • grantee (AWSCDK::IAM::IGrantable) —
    • The IAM principal to grant permissions to.
  • actions (Array<String>) —
    • The actions to grant.

Returns:



315
316
317
318
319
320
321
 
# File 'bedrock_agent_core/evaluator.rb', line 315

def grant(grantee, *actions)
  Jsii::Type.check_type(grantee, JSON.parse(Base64.strict_decode64("eyJmcW4iOiJhd3MtY2RrLWxpYi5hd3NfaWFtLklHcmFudGFibGUifQ==")), "grantee")
  actions.each_with_index do |item, index|
    Jsii::Type.check_type(item, JSON.parse(Base64.strict_decode64("eyJwcmltaXRpdmUiOiJzdHJpbmcifQ==")), "actions[#{index}]")
  end
  jsii_call_method("grant", [grantee, *actions])
end

#node ⇒ Constructs::Node

The tree node.

Returns:

  • (Constructs::Node) 


122
123
124
 
# File 'bedrock_agent_core/evaluator.rb', line 122

def node()
  jsii_get_property("node")
end

#physical_name ⇒ String

Returns a string-encoded token that resolves to the physical name that should be passed to the CloudFormation resource.

This value will resolve to one of the following:

  • a concrete value (e.g. "my-awesome-bucket")
  • undefined, when a name should be generated by CloudFormation
  • a concrete name generated automatically during synthesis, in cross-environment scenarios.

Returns:

  • (String) 


151
152
153
 
# File 'bedrock_agent_core/evaluator.rb', line 151

def physical_name()
  jsii_get_property("physicalName")
end

#stack ⇒ AWSCDK::Stack

The stack in which this resource is defined.

Returns:



158
159
160
 
# File 'bedrock_agent_core/evaluator.rb', line 158

def stack()
  jsii_get_property("stack")
end

#status ⇒ String?

The lifecycle status of the evaluator.

Returns:

  • (String, nil) 


200
201
202
 
# File 'bedrock_agent_core/evaluator.rb', line 200

def status()
  jsii_get_property("status")
end

#to_string ⇒ String

Returns a string representation of this construct.

Returns:

  • (String) 


221
222
223
 
# File 'bedrock_agent_core/evaluator.rb', line 221

def to_string()
  jsii_call_method("toString", [])
end

#updated_at ⇒ String?

The timestamp when the evaluator was last updated.

Returns:

  • (String, nil) 


207
208
209
 
# File 'bedrock_agent_core/evaluator.rb', line 207

def updated_at()
  jsii_get_property("updatedAt")
end

#with(*mixins) ⇒ Constructs::IConstruct

Applies one or more mixins to this construct.

Mixins are applied in order. The list of constructs is captured at the start of the call, so constructs added by a mixin will not be visited. Use multiple with() calls if subsequent mixins should apply to added constructs.

Parameters:

  • mixins (Array<Constructs::IMixin>)

Returns:

  • (Constructs::IConstruct) 


234
235
236
237
238
239
 
# File 'bedrock_agent_core/evaluator.rb', line 234

def with(*mixins)
  mixins.each_with_index do |item, index|
    Jsii::Type.check_type(item, JSON.parse(Base64.strict_decode64("eyJmcW4iOiJjb25zdHJ1Y3RzLklNaXhpbiJ9")), "mixins[#{index}]")
  end
  jsii_call_method("with", [*mixins])
end