Feature engineering methodology for congestion forecasting

Short-term traffic forecasting is a key element in proactive traffic management, e.g., mitigating the negative effect of impending congestion through appropriate capacity allocation at signalized intersections. In this study, we develop a data-driven methodology for reliably and robustly predicting impending stable congestion. By incorporating feature engineering techniques into an iterative machine learning process, we develop a prediction model that can be intuitively understood by traffic experts and is amenable to diagnostics during implementation. Our iterative machine learning process combines the embedded and filter approaches for feature selection with the use of expert knowledge to create aggregative input variables. The embedded approach is represented by application of a decision tree algorithm, while the filter approach is reflected in use of the mean decrease in accuracy output of a random forest algorithm for identifying expressive variables. We tested the methodology by applying it to field data from a sub-network in Tel Aviv. We demonstrated a reduction in the number of decision tree input variables from 66 raw variables to the five most effective aggregative ones, while achieving statistically significant improvement in all performance indicators. The identification rate of stable congestion increased from 65% to 74% while the robustness of the results was enhanced: the standard deviations of the identification and false alarm rates fell from 8% to 3%, respectively, to 5% and 2%.