Class: AWSCDK::KinesisFirehose::CfnDeliveryStream::ParquetSerDeProperty

Inherits:
Jsii::Struct
  • Object
show all
Defined in:
kinesis_firehose/cfn_delivery_stream.rb

Overview

A serializer to use for converting data to the Parquet format before storing it in Amazon S3.

For more information, see Apache Parquet .

See Also:

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(block_size_bytes: nil, compression: nil, enable_dictionary_compression: nil, max_padding_bytes: nil, page_size_bytes: nil, writer_version: nil) ⇒ ParquetSerDeProperty

Returns a new instance of ParquetSerDeProperty.

Parameters:

  • block_size_bytes (Numeric, nil) (defaults to: nil) —

    The Hadoop Distributed File System (HDFS) block size.

  • compression (String, nil) (defaults to: nil) —

    The compression code to use over data blocks.

  • enable_dictionary_compression (Boolean, AWSCDK::IResolvable, nil) (defaults to: nil) —

    Indicates whether to enable dictionary compression.

  • max_padding_bytes (Numeric, nil) (defaults to: nil) —

    The maximum amount of padding to apply.

  • page_size_bytes (Numeric, nil) (defaults to: nil) —

    The Parquet page size.

  • writer_version (String, nil) (defaults to: nil) —

    Indicates the version of row format to output.



3622
3623
3624
3625
3626
3627
3628
3629
3630
3631
3632
3633
3634
3635
 
# File 'kinesis_firehose/cfn_delivery_stream.rb', line 3622

def initialize(block_size_bytes: nil, compression: nil, enable_dictionary_compression: nil, max_padding_bytes: nil, page_size_bytes: nil, writer_version: nil)
  @block_size_bytes = block_size_bytes
  Jsii::Type.check_type(@block_size_bytes, JSON.parse(Base64.strict_decode64("eyJwcmltaXRpdmUiOiJudW1iZXIifQ==")), "blockSizeBytes") unless @block_size_bytes.nil?
  @compression = compression
  Jsii::Type.check_type(@compression, JSON.parse(Base64.strict_decode64("eyJwcmltaXRpdmUiOiJzdHJpbmcifQ==")), "compression") unless @compression.nil?
  @enable_dictionary_compression = enable_dictionary_compression
  Jsii::Type.check_type(@enable_dictionary_compression, JSON.parse(Base64.strict_decode64("eyJ1bmlvbiI6eyJ0eXBlcyI6W3sicHJpbWl0aXZlIjoiYm9vbGVhbiJ9LHsiZnFuIjoiYXdzLWNkay1saWIuSVJlc29sdmFibGUifV19fQ==")), "enableDictionaryCompression") unless @enable_dictionary_compression.nil?
  @max_padding_bytes = max_padding_bytes
  Jsii::Type.check_type(@max_padding_bytes, JSON.parse(Base64.strict_decode64("eyJwcmltaXRpdmUiOiJudW1iZXIifQ==")), "maxPaddingBytes") unless @max_padding_bytes.nil?
  @page_size_bytes = page_size_bytes
  Jsii::Type.check_type(@page_size_bytes, JSON.parse(Base64.strict_decode64("eyJwcmltaXRpdmUiOiJudW1iZXIifQ==")), "pageSizeBytes") unless @page_size_bytes.nil?
  @writer_version = writer_version
  Jsii::Type.check_type(@writer_version, JSON.parse(Base64.strict_decode64("eyJwcmltaXRpdmUiOiJzdHJpbmcifQ==")), "writerVersion") unless @writer_version.nil?
end

Instance Attribute Details

#block_size_bytes ⇒ Numeric? (readonly)

The Hadoop Distributed File System (HDFS) block size.

This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 256 MiB and the minimum is 64 MiB. Firehose uses this value for padding calculations.

Returns:

  • (Numeric, nil)

See Also:



3643
3644
3645
 
# File 'kinesis_firehose/cfn_delivery_stream.rb', line 3643

def block_size_bytes
  @block_size_bytes
end

#compression ⇒ String? (readonly)

The compression code to use over data blocks.

The possible values are UNCOMPRESSED , SNAPPY , and GZIP , with the default being SNAPPY . Use SNAPPY for higher decompression speed. Use GZIP if the compression ratio is more important than speed.

Returns:

  • (String, nil)

See Also:



3650
3651
3652
 
# File 'kinesis_firehose/cfn_delivery_stream.rb', line 3650

def compression
  @compression
end

#enable_dictionary_compression ⇒ Boolean, ... (readonly)

Indicates whether to enable dictionary compression.

Returns:

See Also:



3655
3656
3657
 
# File 'kinesis_firehose/cfn_delivery_stream.rb', line 3655

def enable_dictionary_compression
  @enable_dictionary_compression
end

#max_padding_bytes ⇒ Numeric? (readonly)

The maximum amount of padding to apply.

This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The default is 0.

Returns:

  • (Numeric, nil)

See Also:



3662
3663
3664
 
# File 'kinesis_firehose/cfn_delivery_stream.rb', line 3662

def max_padding_bytes
  @max_padding_bytes
end

#page_size_bytes ⇒ Numeric? (readonly)

The Parquet page size.

Column chunks are divided into pages. A page is conceptually an indivisible unit (in terms of compression and encoding). The minimum value is 64 KiB and the default is 1 MiB.

Returns:

  • (Numeric, nil)

See Also:



3669
3670
3671
 
# File 'kinesis_firehose/cfn_delivery_stream.rb', line 3669

def page_size_bytes
  @page_size_bytes
end

#writer_version ⇒ String? (readonly)

Indicates the version of row format to output.

The possible values are V1 and V2 . The default is V1 .

Returns:

  • (String, nil)

See Also:



3676
3677
3678
 
# File 'kinesis_firehose/cfn_delivery_stream.rb', line 3676

def writer_version
  @writer_version
end

Class Method Details

.jsii_properties ⇒ Object 



3678
3679
3680
3681
3682
3683
3684
3685
3686
3687
 
# File 'kinesis_firehose/cfn_delivery_stream.rb', line 3678

def self.jsii_properties
  {
    :block_size_bytes => "blockSizeBytes",
    :compression => "compression",
    :enable_dictionary_compression => "enableDictionaryCompression",
    :max_padding_bytes => "maxPaddingBytes",
    :page_size_bytes => "pageSizeBytes",
    :writer_version => "writerVersion",
  }
end

Instance Method Details

#to_jsii ⇒ Object 



3689
3690
3691
3692
3693
3694
3695
3696
3697
3698
3699
3700
 
# File 'kinesis_firehose/cfn_delivery_stream.rb', line 3689

def to_jsii
  result = {}
  result.merge!({
    "blockSizeBytes" => @block_size_bytes,
    "compression" => @compression,
    "enableDictionaryCompression" => @enable_dictionary_compression,
    "maxPaddingBytes" => @max_padding_bytes,
    "pageSizeBytes" => @page_size_bytes,
    "writerVersion" => @writer_version,
  })
  result.compact
end