← Home Dark mode

Hotel Booking Cancellation Prediction

Machine Learning Project Report

This report documents the workflow from exploratory analysis and data preparation to model selection, evaluation, threshold selection and production scoring.

1. Introduction and Business Problem

Hotel booking cancellations create operational and financial challenges because rooms reserved for customers who later cancel may remain unsold. Identifying bookings with a high cancellation risk can support better reservation management, confirmation follow-up, inventory planning, and revenue-related decisions.

The objective of this project is to develop a machine-learning model that estimates the probability that a hotel booking will be cancelled. Historical booking records from two hotels were combined with guest information, while payment-event data was investigated separately to determine whether it could provide valid predictors at the booking-time prediction point.

A non-machine-learning heuristic was first established as a baseline. Logistic Regression, Random Forest, and HistGradientBoosting classifiers were then developed and compared using a chronological train-test strategy.

1.1 Project Objective

The primary objective is to develop a reliable cancellation-risk model using information available at or before the prediction point. Performance is assessed using accuracy, precision, recall, F1-score, ROC-AUC and Average Precision.

2. Dataset and Data Sources

2.1 Booking Data

The primary booking data consists of H1.csv and H2.csv. The two files share a common structure and were combined after adding a Hotel variable to preserve the source hotel.

2.2 Guest Data

The guests.csv file contains guest-level information including guest ID, name, age, country, email and notes. Duplicate guest IDs were consolidated, and age was retained as a modelling feature. Personally identifying fields such as names and email addresses were excluded.

2.3 Payment Event Data

The payment log contained payment_received, card_declined, deposit_authorised and refund_issued events. Timing analysis showed that payment events could not reliably be proven to be available before the booking-time prediction point. Payment-derived features were therefore excluded to avoid data leakage.

2.4 Target Variable

IsCanceled is the binary target, where 0 represents a non-cancelled booking and 1 represents a cancelled booking.

3. Exploratory Data Analysis

Exploratory Data Analysis (EDA) was used not only to describe the datasets, but also to identify data-quality problems, examine variables associated with cancellation, and decide which information could legitimately be available when a booking is scored. This was particularly important because several variables appeared highly predictive at first sight but could have introduced future information into the model.

3.1 Target Distribution

After the main cleaning decisions, the modelling dataset contained 119,210 bookings. Of these, 75,011 (62.9%) were not cancelled and 44,199 (37.1%) were cancelled. The target was therefore moderately imbalanced: cancellation was less common, but not rare. Consequently, model comparison was not based on accuracy alone. Precision, recall, F1-score and probability-based measures were also considered because they provide more information about performance on the cancellation class.

3.2 Lead Time

Lead time showed a clear relationship with cancellation behaviour. Bookings made further in advance tended to have higher cancellation rates than bookings made close to arrival. This is operationally plausible because a longer period between reservation and arrival gives more opportunity for travel plans, prices or customer circumstances to change.

The relationship was useful in two ways. First, it supported retaining LeadTime as a model feature. Second, it provided a simple and explainable variable for the non-ML baseline, where a lead time greater than 150 days was treated as one indication of cancellation risk. The machine-learning models were still allowed to learn more complex relationships rather than being restricted to this single cutoff.

3.3 Deposit Type

Deposit type produced one of the strongest descriptive differences in cancellation behaviour. In the combined booking data, Non Refund bookings showed an unusually high observed cancellation rate, whereas No Deposit bookings had a much lower rate. This finding was retained as an empirical relationship in the dataset rather than interpreted as proof that the deposit itself caused cancellation.

Because DepositType is part of the original booking information, it is available at prediction time and can be used without introducing leakage. It was therefore retained as a categorical model feature and used, together with lead time, to construct the heuristic baseline.

3.4 ADR and Booking-Record Quality

Average Daily Rate (ADR) required explicit investigation. H1 contained one negative ADR observation of -6.38, while H2 contained one extreme value of 5,400. These two observations were exceptional relative to the rest of the distributions. Rather than defining an arbitrary universal maximum ADR, the two specific anomalous observations identified during EDA were treated and the remaining distribution was rechecked. After treatment, ADR ranged from 0 to 510, with a median of approximately 94.95.

Additional integrity checks identified 180 zero-guest bookings. These records did not represent meaningful occupied bookings and were removed. In contrast, zero-night bookings were retained because most were not cancelled and there was insufficient evidence to classify them as errors; they may represent day-use or administrative booking patterns.

3.5 Guest Profile and Repeat-Booking Behaviour

The guest file contained 107,896 rows but only 106,828 unique guest IDs, indicating 1,068 duplicated IDs. Every duplicated ID appeared exactly twice. Sample inspection showed that the duplicate rows generally represented the same person, with differences such as name capitalisation or a missing age. No conflicting non-null ages or countries were found for these duplicated IDs, so the records could be consolidated safely.

Age also contained quality problems. After guest consolidation there were 1,050 missing ages, while the original age values included 235 observations below zero and 329 above 100, with a maximum of 999. Values outside the 0–100 range were treated as invalid and, together with missing ages, imputed using the median valid age of 46.

The booking-level country and guest-profile country matched for approximately 99.60% of bookings. Because the two variables carried almost the same information, the guest-profile country was excluded and the booking-level Country field was retained to avoid redundant features.

3.6 Payment and Weather Data Decisions

The payment log initially appeared promising because events such as card declines and deposit authorisations were associated with cancellation. A timing check, however, showed that the event information was not safely available at the intended booking-time prediction point. In the full raw payment log, 107,183 events occurred after the calculated booking date and 12,601 occurred on the same calendar day. Because the booking dataset did not contain an exact booking timestamp, the temporal order of same-day events could not be established. Payment-event variables were therefore excluded from the final predictive features to avoid leakage.

Weather data was also investigated. Location labels required standardisation and the data contained questionable observations, including implausible temperature values. More importantly, actual weather on a future arrival date would not be known when many bookings were made. Actual arrival-day weather was therefore excluded from the booking-time model rather than giving the model information that would only become available later.

4. Data Cleaning and Feature Engineering

The cleaning pipeline was designed to be reproducible: the raw files were loaded again in the dedicated cleaning notebook and each transformation was applied explicitly before the processed modelling dataset was saved. The main objective was not simply to remove unusual records, but to produce a consistent feature set while preserving valid booking behaviour and preventing data leakage.

4.1 Combining and Cleaning Booking Data

H1.csv and H2.csv were combined after adding a Hotel indicator so that differences between the two properties could still be learned by the model. Missing values in Children were handled, the identified ADR anomalies were replaced using the median ADR for the corresponding hotel, and zero-occupancy bookings were excluded. Fields used only for cleaning or chronological processing were kept during feature construction and removed from the final predictor matrix when no longer required.

4.2 Guest Data Consolidation

Duplicate guest records were consolidated to one row per guest_id. For age and country, groupby() aggregation was used to preserve available non-null information. Invalid and missing ages were then imputed using the valid median age of 46. Names, email addresses and free-text notes were not used as predictors because they were personally identifying or unstructured and were not required for the cancellation model.

4.3 Date Construction and Prediction-Time Logic

An ArrivalDate was reconstructed from the year, month and day fields. Because an explicit booking creation date was not provided, an approximate BookingDate was derived as:

BookingDate = ArrivalDate - LeadTime

This derived date was essential for chronological ordering, historical guest features and the train-test split. It was used as a process variable rather than passed directly to the classifier.

4.4 Engineered Booking Features

• TotalNights = StaysInWeekendNights + StaysInWeekNights, giving a direct measure of stay duration.
• TotalGuests = Adults + Children + Babies, giving a single occupancy measure.
• age was added from the cleaned guest profile.

4.5 Leakage-Safe Guest-History Features

Repeat-customer behaviour was represented through three engineered variables: PreviousBookings, PreviousCancellations and PreviousCancellationRate. Bookings were sorted by GuestID and BookingDate. cumcount() was used to count earlier bookings, while the cancellation outcome was shifted before taking the cumulative sum so that the current booking's target was never included in its own historical feature.

PreviousCancellationRate =
PreviousCancellations / PreviousBookings

Guests with no previous bookings were assigned a previous cancellation rate of zero. This feature design is important because a simpler total-per-guest calculation would accidentally include future bookings and would overstate real production performance.

4.6 Final Feature Set and Preprocessing

The final model used 29 predictors: 19 numerical and 10 categorical. Unique identifiers (BookingID and GuestID) and helper dates were excluded from the model input. Categorical variables were one-hot encoded with handle_unknown='ignore', allowing later test or production bookings to contain categories not observed during training. During evaluation, previously unseen values were in fact encountered for Country and Agent, demonstrating why this safeguard was useful.

Preprocessing was encapsulated inside scikit-learn pipelines rather than applied manually to the complete dataset. This ensured that transformations were learned from the training data and then applied consistently to validation, test and production inputs.

5. Non-ML Heuristic Baseline

A non-machine-learning baseline was created before fitting the classifiers. The purpose of this baseline was to establish how much useful predictive performance could be obtained from a simple, transparent business rule and therefore provide a meaningful benchmark for judging whether machine learning added value.

The rule predicted cancellation when either the booking had a Non Refund deposit or LeadTime > 150 days. These conditions were chosen from the EDA because both variables showed clear relationships with cancellation and could be known from the booking record without relying on future outcomes.

Metric	Heuristic Baseline
Accuracy	70.95%
Precision	58.76%
Recall	26.66%
F1-score	36.67%

The baseline's main weakness was recall. It identified only 26.66% of actual cancellations, which means that nearly three quarters of cancellations were not captured by the rule. Accuracy of 70.95% may appear acceptable in isolation, but it hides this poor detection of the positive class. The low F1-score of 36.67% likewise shows that the rule did not provide a useful balance between finding cancellations and keeping cancellation alerts reliable.

This made the baseline valuable as a benchmark: a machine-learning model would need to improve substantially on cancellation recall and F1-score, not merely increase overall accuracy by a small amount.

6. Model Development

The modelling stage compared three classifier families with different assumptions: Logistic Regression as a linear and interpretable benchmark, Random Forest as a bagged tree ensemble, and HistGradientBoostingClassifier as a sequential boosting method capable of learning nonlinear relationships and feature interactions.

6.1 Chronological Train-Test Split

A random train-test split was deliberately avoided for the final evaluation. The derived booking dates ranged from 24 June 2013 to 31 August 2017. After sorting chronologically, bookings before 12 January 2017 were assigned to training and bookings from that date onward to testing. This produced 95,278 training rows and 23,932 test rows, with no date overlap between the two sets.

The training cancellation rate was 38.46%, compared with 31.56% in the later test period. This change is itself informative: the chronological test is more demanding because the future period is not distributed exactly like the historical training period. It therefore better approximates the production question, “Can a model trained on past bookings generalise to later bookings?”

6.2 Logistic Regression

Logistic Regression provided a relatively interpretable linear benchmark. Numerical features were standardised with StandardScaler, while categorical features were one-hot encoded. The model reached 80.25% accuracy and an ROC-AUC of 0.8751. Cancellation recall was 67.36%, showing a major improvement over the heuristic baseline.

6.3 Random Forest

Random Forest was used to capture nonlinear relationships and interactions without requiring numerical standardisation. It improved accuracy to 82.54%, precision to 77.33% and ROC-AUC to 0.8975. Its main trade-off was lower cancellation recall of 63.19%, meaning it generated fewer false cancellation alerts but missed more actual cancellations than Logistic Regression.

6.4 HistGradientBoostingClassifier

HistGradientBoosting builds trees sequentially, with later trees correcting errors made by earlier ones. This model produced the strongest overall initial results: 83.26% accuracy, 69.32% recall, a 72.33% F1-score and 0.9090 ROC-AUC. Although Random Forest retained the highest precision among the untuned models, HistGradientBoosting provided the best overall balance and was therefore selected for tuning.

Model	Accuracy	Precision	Recall	F1	ROC-AUC
Logistic Regression	80.25%	69.22%	67.36%	68.28%	87.51%
Random Forest	82.54%	77.33%	63.19%	69.55%	89.75%
HistGradientBoosting	83.26%	75.61%	69.32%	72.33%	90.90%

7. Hyperparameter Tuning

HistGradientBoostingClassifier was tuned because it produced the strongest overall baseline ML performance. Hyperparameter selection was performed using the training data only. A five-split TimeSeriesSplit cross-validation strategy was used instead of shuffled k-fold validation so that each validation fold occurred later in time than the observations used to train it.

RandomizedSearchCV evaluated 20 parameter combinations and used ROC-AUC as the scoring criterion. ROC-AUC was chosen because the intended production output is a cancellation probability and the metric assesses the model's ability to rank higher-risk bookings above lower-risk bookings across possible thresholds.

7.1 Search Parameters

The search considered learning rate, number of boosting iterations, maximum leaf nodes, minimum samples per leaf and L2 regularisation. The selected configuration was:

min_samples_leaf = 40
max_leaf_nodes = 31
max_iter = 300
learning_rate = 0.1
l2_regularization = 1.0

The best mean time-series cross-validation ROC-AUC was approximately 0.9134. When this selected model was applied to the chronological test period at the default 0.50 class decision, it achieved 83.52% accuracy, 75.71% precision, 70.35% recall, a 72.93% F1-score and 0.9091 ROC-AUC.

The improvement over the untuned HistGradientBoosting model was modest rather than dramatic. The main gain was in cancellation recall, which increased from 69.32% to 70.35%. ROC-AUC remained almost unchanged, suggesting that the original model already ranked cancellation risk effectively and that tuning mainly refined the final decision behaviour.

8. Classification Threshold Selection

The classifier produces a probability for class 1 (cancellation). A final class decision requires an operating threshold. With the conventional 0.50 cutoff, a booking with a predicted cancellation probability of 0.49 is classified as non-cancelled, while a probability of 0.51 is classified as cancelled. Changing the threshold does not retrain the model or change its probabilities; it changes how those probabilities are converted into an operational decision.

Because missing an actual cancellation may prevent the hotel from taking timely confirmation or inventory actions, thresholds from 0.30 to 0.70 were compared to understand the precision-recall trade-off.

Threshold	Accuracy	Precision	Recall	F1
0.30	81.28%	65.75%	84.94%	74.12%
0.40	83.15%	71.52%	77.45%	74.37%
0.50	83.52%	75.71%	70.35%	72.93%
0.60	82.81%	79.42%	61.47%	69.30%
0.70	81.72%	84.30%	51.69%	64.09%

At 0.40, recall increased to 77.45% and the F1-score reached 74.37%, the highest F1-score among the evaluated thresholds. Compared with 0.50, accuracy decreased only slightly (83.52% to 83.15%) while recall increased by 7.10 percentage points. Precision decreased from 75.71% to 71.52%, reflecting the expected cost of flagging more bookings as high risk.

Operationally, the 0.40 threshold therefore favours detecting more genuine cancellations at the expense of some additional false alerts. It should not be interpreted as a permanent property of the model: if the business cost of false positives or missed cancellations changes, the operating threshold can be reassessed without retraining the classifier.

Methodological note: the threshold comparison in this project was performed using the chronological evaluation results. In a stricter future deployment workflow, the operating threshold should be selected on a validation period and fixed before the final test set is evaluated.

9. Final Model Evaluation

The final model is the tuned HistGradientBoostingClassifier operated at the selected 0.40 threshold. Evaluation combines threshold-dependent metrics (accuracy, precision, recall and F1) with probability-ranking metrics (ROC-AUC and Average Precision). This is important because no single metric captures all aspects of the cancellation problem.

Metric	Final Result	Interpretation
Accuracy	83.15%	Overall proportion of test bookings classified correctly.
Precision	71.52%	Of bookings flagged as cancellations, 71.52% actually cancelled.
Recall	77.45%	The model identified 77.45% of all actual cancellations.
F1-score	74.37%	Harmonic balance between cancellation precision and recall.
ROC-AUC	90.91%	Strong overall ability to rank cancelled bookings above non-cancelled bookings.
Average Precision	83.39%	Strong precision-recall performance across probability thresholds.

9.1 Confusion Matrix

The final confusion matrix contained 14,051 true negatives, 2,329 false positives, 1,703 false negatives and 5,849 true positives. In business terms, 5,849 cancellations were correctly flagged, while 1,703 actual cancellations were missed. The 2,329 false positives represent bookings that would receive a high-risk alert even though they ultimately did not cancel.

9.2 ROC Curve

The ROC curve considers the true-positive rate and false-positive rate across many possible thresholds. The model's ROC-AUC of 0.9091 is independent of the selected 0.40 operating threshold because it is calculated from the predicted probabilities themselves. A useful interpretation is that a randomly selected cancelled booking will usually receive a higher model risk score than a randomly selected non-cancelled booking.

9.3 Precision-Recall Curve

The Precision-Recall curve focuses directly on the positive cancellation class and illustrates how precision changes as recall is increased through different thresholds. The Average Precision score of 0.8339 summarises this relationship and supports the conclusion that the model maintains useful precision while retrieving a substantial proportion of actual cancellations.

9.4 Improvement over the Heuristic Baseline

The final comparison demonstrates why the machine-learning approach was justified. Accuracy improved from 70.95% to 83.15%, precision from 58.76% to 71.52%, and F1-score from 36.67% to 74.37%. The most substantial improvement was recall, which increased from 26.66% to 77.45%—an improvement of 50.79 percentage points.

This means the ML model captured cancellation patterns that could not be represented adequately by the simple lead-time/deposit rule. The gain is therefore not merely statistical; it directly affects the number of genuine high-risk bookings that can be surfaced for operational attention.

10. Production Scoring

The final fitted HistGradientBoosting pipeline was saved using Joblib so that predictions can be made on new bookings without retraining the model. Metadata was stored separately to record the model name, target class, selected decision threshold, and the minimum and maximum lead times observed in the training data.

{
  "model": "HistGradientBoostingClassifier",
  "threshold": 0.4,
  "target": "IsCanceled",
  "positive_class": 1,
  "lead_time_min": 0,
  "lead_time_max": 737
}

10.1 Production Scoring Process

The production scoring script accepts new booking information and prepares it in the same structure used during model development. Required engineered features are calculated from the supplied booking information, while historical guest features are retrieved from the available guest and booking history where applicable.

The prepared booking is then passed to the saved machine-learning pipeline. The model uses predict_proba() to estimate the probability that the booking will be cancelled. The selected classification threshold of 0.40 is then applied to convert this probability into an operational risk decision. A probability equal to or greater than 0.40 is classified as high cancellation risk, while a probability below 0.40 is classified as low cancellation risk.

A typical scoring response is shown below:

{
  "cancellation_probability": 0.0691,
  "prediction": 0,
  "decision": "Low cancellation risk",
  "threshold": 0.4,
  "warnings": []
}

~ In this example, the estimated cancellation probability is 6.91%. Since this is below the 0.40 decision threshold, the booking is classified as Low cancellation risk.

10.2 Input Validation and Reliability Checks

Basic input validation was added to the production scoring tool to reduce invalid inputs and identify bookings that fall outside the range represented in the training data. This is important because a machine-learning model may still return a probability for an unusual input even when that value is substantially different from the observations on which the model was trained.

The minimum and maximum lead times observed during model training are stored in the model metadata as lead_time_min and lead_time_max. For the current model, these values are 0 and 737 days, respectively. The production application reads these values from the metadata and uses them when validating new bookings.

lead_time_min = metadata["lead_time_min"]
lead_time_max = metadata["lead_time_max"]

A negative lead time is rejected because it would indicate that the arrival date occurs before the booking date. If a booking has a lead time greater than 737 days, the booking can still be scored, but the application returns a warning indicating that the value is outside the range observed during model training. The resulting prediction should therefore be interpreted with greater caution.

The 0–737 day range is not a restriction on hotel reservations. It represents the range of lead times observed in the training data and is used only as a reliability check. Storing these limits in the model metadata also means that the validation logic can use updated limits if the model is retrained on data with a different lead-time range.

For example, an out-of-range booking may return:

{
  "cancellation_probability": 0.2133,
  "prediction": 0,
  "decision": "Low cancellation risk",
  "threshold": 0.4,
  "warnings": [
    "Lead time is outside the range observed during model training (0-737 days). Prediction may be less reliable."
  ]
}

The 0–737 day range is not a restriction on hotel reservations. It represents the range of lead times observed during model training and is therefore used only as a reliability check for new predictions.

10.3 Lightweight Web Application

A lightweight local web application was developed to demonstrate how the saved model could be used as a production risk-assessment tool. The application uses a simple HTML and JavaScript frontend and a Python backend based on Python's standard HTTP server functionality. No additional Python web framework is required.

The user enters normal booking information through the web form. Where possible, controlled inputs such as dropdown lists are used for categorical variables including meal type, deposit type, market segment, distribution channel, room type, customer type and hotel. This reduces the possibility of invalid categorical values being submitted to the model.

Booking and arrival dates can also be used to derive model inputs automatically. For example, LeadTime is calculated as the number of days between the booking date and arrival date, while the arrival year, month, week number and day of month can be derived from the selected arrival date. This prevents inconsistent combinations of date-related inputs from being entered manually.

10.4 Production Workflow

The local production workflow can be summarised as follows:

This implementation allows the trained model to be tested locally without modifying or retraining it. A user can start the Python server, open the web interface, enter a booking and obtain a cancellation probability and risk classification from the saved model.

10.5 Operational Interpretation

The prediction is intended to support hotel decision-making rather than automatically determine what should happen to a reservation. A booking classified as high cancellation risk could, for example, be prioritised for confirmation, deposit or payment follow-up, closer monitoring, or inventory-planning decisions.

Predictions should therefore be treated as risk indicators rather than guaranteed outcomes. Even a booking with a high predicted probability may ultimately stay, while some bookings classified as low risk may still cancel.

11. Monitoring, Retraining and Business Actions

Model performance should be monitored as new outcomes become available. Accuracy, precision, recall, F1-score, ROC-AUC and the distribution of predicted probabilities should be reassessed periodically.

Retraining should be considered when predictive performance deteriorates or when booking behaviour, channels, pricing or customer mix changes materially.

High-risk predictions may support confirmation reminders, deposit follow-up, closer monitoring and inventory planning. Predictions should support human decisions rather than automatically cancelling reservations.

12. Limitations

• The model does not identify every cancellation.
• False cancellation alerts remain possible.
• Payment events were excluded because their timing could introduce leakage.
• Guest age required cleaning and imputation.
• Previously unseen categorical values may appear over time.
• The selected threshold should be reviewed if business costs change.

13. Conclusion

This project developed a machine-learning solution for hotel booking cancellation prediction. The tuned HistGradientBoostingClassifier, evaluated on a later chronological test period, achieved 83.15% accuracy, 71.52% precision, 77.45% recall, a 74.37% F1-score and a 0.9091 ROC-AUC at the selected 0.40 threshold.

The final model substantially outperformed the simple heuristic baseline and was successfully incorporated into a production scoring workflow that returns both cancellation probability and an operational risk decision. Continued monitoring and periodic retraining will be required to maintain performance as booking behaviour changes.