Type Alias SegmentProcessing

SegmentProcessing: {
    Caption?: AutoGenerationConfig | null;
    Footnote?: AutoGenerationConfig | null;
    Formula?: LlmGenerationConfig | null;
    ListItem?: AutoGenerationConfig | null;
    Page?: LlmGenerationConfig | null;
    PageFooter?: AutoGenerationConfig | null;
    PageHeader?: AutoGenerationConfig | null;
    Picture?: PictureGenerationConfig | null;
    SectionHeader?: AutoGenerationConfig | null;
    Table?: LlmGenerationConfig | null;
    Text?: AutoGenerationConfig | null;
    Title?: AutoGenerationConfig | null;
}

Controls the post-processing of each segment type.

Allows you to generate HTML and Markdown from chunkr models for each segment type. By default, the HTML and Markdown are generated manually using the segmentation information except for Table, Formula and Picture. You can optionally configure custom LLM prompts and models to generate an additional llm field with LLM-processed content for each segment type.

The configuration of which content sources (HTML, Markdown, LLM, Content) of the segment should be included in the chunk's embed field and counted towards the chunk length can be configured through the embed_sources setting.

Type Alias SegmentProcessing

Settings