Class: AWSCDK::KinesisFirehose::ParquetOutputFormatProps

Inherits:
Jsii::Struct
  • Object
show all
Defined in:
kinesis_firehose/parquet_output_format_props.rb

Overview

Props for Parquet output format for data record format conversion.

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(block_size: nil, compression: nil, enable_dictionary_compression: nil, max_padding: nil, page_size: nil, writer_version: nil) ⇒ ParquetOutputFormatProps

Returns a new instance of ParquetOutputFormatProps.

Parameters:

  • block_size (AWSCDK::Size, nil) (defaults to: nil) —

    The Hadoop Distributed File System (HDFS) block size.

  • compression (AWSCDK::KinesisFirehose::ParquetCompression, nil) (defaults to: nil) —

    The compression code to use over data blocks.

  • enable_dictionary_compression (Boolean, nil) (defaults to: nil) —

    Indicates whether to enable dictionary compression.

  • max_padding (AWSCDK::Size, nil) (defaults to: nil) —

    The maximum amount of padding to apply.

  • page_size (AWSCDK::Size, nil) (defaults to: nil) —

    The Parquet page size.

  • writer_version (AWSCDK::KinesisFirehose::ParquetWriterVersion, nil) (defaults to: nil) —

    Indicates the version of Parquet to output.



13
14
15
16
17
18
19
20
21
22
23
24
25
26
 
# File 'kinesis_firehose/parquet_output_format_props.rb', line 13

def initialize(block_size: nil, compression: nil, enable_dictionary_compression: nil, max_padding: nil, page_size: nil, writer_version: nil)
  @block_size = block_size
  Jsii::Type.check_type(@block_size, JSON.parse(Base64.strict_decode64("eyJmcW4iOiJhd3MtY2RrLWxpYi5TaXplIn0=")), "blockSize") unless @block_size.nil?
  @compression = compression
  Jsii::Type.check_type(@compression, JSON.parse(Base64.strict_decode64("eyJmcW4iOiJhd3MtY2RrLWxpYi5hd3Nfa2luZXNpc2ZpcmVob3NlLlBhcnF1ZXRDb21wcmVzc2lvbiJ9")), "compression") unless @compression.nil?
  @enable_dictionary_compression = enable_dictionary_compression
  Jsii::Type.check_type(@enable_dictionary_compression, JSON.parse(Base64.strict_decode64("eyJwcmltaXRpdmUiOiJib29sZWFuIn0=")), "enableDictionaryCompression") unless @enable_dictionary_compression.nil?
  @max_padding = max_padding
  Jsii::Type.check_type(@max_padding, JSON.parse(Base64.strict_decode64("eyJmcW4iOiJhd3MtY2RrLWxpYi5TaXplIn0=")), "maxPadding") unless @max_padding.nil?
  @page_size = page_size
  Jsii::Type.check_type(@page_size, JSON.parse(Base64.strict_decode64("eyJmcW4iOiJhd3MtY2RrLWxpYi5TaXplIn0=")), "pageSize") unless @page_size.nil?
  @writer_version = writer_version
  Jsii::Type.check_type(@writer_version, JSON.parse(Base64.strict_decode64("eyJmcW4iOiJhd3MtY2RrLWxpYi5hd3Nfa2luZXNpc2ZpcmVob3NlLlBhcnF1ZXRXcml0ZXJWZXJzaW9uIn0=")), "writerVersion") unless @writer_version.nil?
end

Instance Attribute Details

#block_size ⇒ AWSCDK::Size? (readonly)

Note:

Default: Size.mebibytes(256)

The Hadoop Distributed File System (HDFS) block size.

This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. Firehose uses this value for padding calculations.

Returns:



35
36
37
 
# File 'kinesis_firehose/parquet_output_format_props.rb', line 35

def block_size
  @block_size
end

#compression ⇒ AWSCDK::KinesisFirehose::ParquetCompression? (readonly)

Note:

Default: SNAPPY

The compression code to use over data blocks.

The possible values are UNCOMPRESSED , SNAPPY , and GZIP. Use SNAPPY for higher decompression speed. Use GZIP if the compression ratio is more important than speed.

Returns:

See Also:



45
46
47
 
# File 'kinesis_firehose/parquet_output_format_props.rb', line 45

def compression
  @compression
end

#enable_dictionary_compression ⇒ Boolean? (readonly)

Note:

Default: false

Indicates whether to enable dictionary compression.

Returns:

  • (Boolean, nil)

See Also:



51
52
53
 
# File 'kinesis_firehose/parquet_output_format_props.rb', line 51

def enable_dictionary_compression
  @enable_dictionary_compression
end

#max_padding ⇒ AWSCDK::Size? (readonly)

Note:

Default: no padding is applied

The maximum amount of padding to apply.

This is useful if you intend to copy the data from Amazon S3 to HDFS before querying.

Returns:

See Also:



59
60
61
 
# File 'kinesis_firehose/parquet_output_format_props.rb', line 59

def max_padding
  @max_padding
end

#page_size ⇒ AWSCDK::Size? (readonly)

Note:

Default: Size.mebibytes(1)

The Parquet page size.

Column chunks are divided into pages. A page is conceptually an indivisible unit (in terms of compression and encoding). The minimum value is 64 KiB and the default is 1 MiB.

Returns:

See Also:



67
68
69
 
# File 'kinesis_firehose/parquet_output_format_props.rb', line 67

def page_size
  @page_size
end

#writer_version ⇒ AWSCDK::KinesisFirehose::ParquetWriterVersion? (readonly)

Note:

Default: V1

Indicates the version of Parquet to output.

The possible values are V1 and V2

Returns:

See Also:



75
76
77
 
# File 'kinesis_firehose/parquet_output_format_props.rb', line 75

def writer_version
  @writer_version
end

Class Method Details

.jsii_properties ⇒ Object 



77
78
79
80
81
82
83
84
85
86
 
# File 'kinesis_firehose/parquet_output_format_props.rb', line 77

def self.jsii_properties
  {
    :block_size => "blockSize",
    :compression => "compression",
    :enable_dictionary_compression => "enableDictionaryCompression",
    :max_padding => "maxPadding",
    :page_size => "pageSize",
    :writer_version => "writerVersion",
  }
end

Instance Method Details

#to_jsii ⇒ Object 



88
89
90
91
92
93
94
95
96
97
98
99
 
# File 'kinesis_firehose/parquet_output_format_props.rb', line 88

def to_jsii
  result = {}
  result.merge!({
    "blockSize" => @block_size,
    "compression" => @compression,
    "enableDictionaryCompression" => @enable_dictionary_compression,
    "maxPadding" => @max_padding,
    "pageSize" => @page_size,
    "writerVersion" => @writer_version,
  })
  result.compact
end