LLM-Guided Blockwise Feature Engineering Turns Long Text into Executable Feature Programs
ORIGINAL / Long Text to Predictive Features: LLM-Guided Blockwise Feature Engineering via Executable Program Search
This work proposes LLM-BlockFE, a framework that offline-converts long text into executable feature programs so that online inference does not require LLM calls. Its value lies in using long-text information while supporting industrial risk-control deployment, with AUC improvements reported on public and private datasets.
01 ABSTRACT
Industrial risk-control systems typically rely on structured-data models, yet substantial valuable information remains embedded in unstructured long text; manual feature engineering is labor-intensive, while calling an LLM for every online input may not meet deployment requirements. The authors propose LLM-BlockFE, which constructs executable feature programs by incrementally appending immutable code blocks and evaluating candidate features with a downstream model. It introduces a block-level rollback mechanism based on depth-calibrated credit allocation and advances multiple independent search trajectories in an interleaved manner to reduce redundant exploration.
02 KEY FINDINGS
- LLM-BlockFE offline-converts long text into executable feature programs that are frozen and deployed, so online inference avoids LLM calls.
- A block-level rollback mechanism with depth-calibrated credit allocation mitigates conventional greedy search getting trapped in suboptimal solutions.
- Multiple independent search trajectories are advanced in an interleaved manner, sharing fixed descriptions of each trajectory's exploration direction to reduce redundant exploration.
AI GENERATED SUMMARY / DISCOVERED BY ARXIV CS.CL · ARXIV CS.LG