SELF-SUPERVISED FOUNDATION MODELS FOR REAL-TIME DRILLING OPTIMIZATION: CROSS-MODAL KNOWLEDGE TRANSFER FROM WELL LOG REPRESENTATIONS

Аңдатпа

The optimization of real-time drilling faces significant hurdles stemming from limited data availability, complex operational environments, and a lack of machine learning models that can transfer knowledge across different geological settings. Current supervised methods deliver strong results in predicting rate of penetration and detecting stuck pipe incidents, yet they demand extensive labeled datasets for each new well and fail to tap into the geological insights contained in commonly available well log measurements. We present DrillFM, a self-supervised foundation model designed for multivariate drilling time-series data that incorporates geological information through cross-modal conditioning from pretrained well log representations. Our architecture merges masked autoencoding of drilling parameters-including weight-on-bit, rotary speed, torque, standpipe pressure, and flow rate-with a cross-modal attention mechanism that draws upon formation embeddings generated by a physics-aware well log foundation model. The training employs a drilling-adaptive curriculum that escalates task complexity progressively: beginning with point-wise masking, advancing to channel-wise masking, and culminating in interval-based prediction. On the publicly available Volve field dataset plus proprietary multi-basin data, geological conditioning cuts rate of penetration prediction error by 18.3 percent relative to the supervised multi-modal baseline. Few-shot transfer to previously unseen wells achieves comparable performance using 90 percent less labeled data. In cross-basin transfer of rate-of-penetration prediction, DrillFM's performance degrades by 38.9 percent moving from Norwegian Volve to Dutch North Sea wells, 13.0 percentage points less than the supervised multi-modal baseline's 51.9 percent degradation. The model delivers inference speeds of 8.6 milliseconds per depth sample using precomputed well log embeddings; because the reported results condition on log data from the same well being predicted, whether this speed and accuracy hold under fully real-time deployment-where only offset-well or no logs would be available-has not been separately evaluated (see Limitations). Ablation experiments show that well log foundation model pretraining is the single largest contributor, producing a 17.7 percent error increase when removed; removing geological conditioning overall accounts for a 12.5 percentage point change, and the masking curriculum contributes a further 5.7 points.