Data Science: A Complete Guide to Methods and Modelling

A useful answer to “what is data science” is a workflow rather than a list of technologies. Data scientists translate a practical question into measurable outcomes, establish whether available data can support an answer and choose appropriate analytical methods. Those methods may include descriptive summaries, statistical estimation, predictive modelling or controlled experiments. Not every problem needs machine learning: a reliable rate estimate or a well-designed comparison may be more useful than a complex model. The defining requirement is that the analysis supports a defensible decision, with assumptions, uncertainty and limitations made clear to the people who will use its results.

The comparisons data science vs data analytics and data science vs machine learning describe overlapping areas, not rigid boundaries. Data analytics often emphasises understanding performance, explaining changes and communicating findings, although analysts also forecast and experiment. Data science commonly extends into building and maintaining predictive systems and evaluating interventions. Machine learning is a family of methods that learns patterns from examples; it is one component of data science, not a replacement for statistical reasoning or experimental design. These distinctions matter because selecting the right approach depends on the decision and evidence, rather than on the job title attached to it.

Begin with the operational question: which active subscribers are likely to cancel within the next 30 days? Define one row per customer at a scoring date, restrict predictors to information already available then, and wait until the outcome window closes before labelling each row. The example below generates independent customer snapshots with tenure, historical usage and support contacts. Missing usage values represent incomplete measurement. Install NumPy, pandas and scikit-learn in a Python environment, then run the snippets in order. The fixed seed makes this demonstration reproducible; the sample size and coefficients are illustrative choices, not claims about any subscription business. ```python import numpy as np import pandas as pd rng=np.random.default_rng(42) n=1500 X=pd.DataFrame({ 'tenure':rng.integers(1,49,n), 'usage':rng.gamma(2,4,n), 'tickets':rng.poisson(1.5,n) }) z=-0.4-0.035*X.tenure-0.12*X.usage+0.45*X.tickets y=pd.Series(rng.binomial(1,1/(1+np.exp(-z))),name='churn') X.loc[rng.random(n)<0.08,'usage']=np.nan ```

Before modelling real data, document the observation unit, eligibility rules, scoring time and target horizon. A cancellation timestamp belongs in target construction, not among predictors for a decision made earlier. Likewise, a support ticket closed after scoring cannot be treated as historical information merely because it appears in today's database export. Check whether records can reconstruct what was known at the time, including publication delays and later corrections. For subscription data, also distinguish cancellation requests from completed departures and account for customers whose observation window has not finished. These decisions determine what the model learns and whether its evaluation is credible.

Statistics for data science starts with understanding how observations were collected. Check missingness, plausible ranges, duplicated identifiers, outcome prevalence and subgroup coverage. Ask whether the dataset excludes customers who never activated, includes only successful payments or overrepresents a particular acquisition channel. Such selection can limit where conclusions apply. Associations also require careful interpretation: customers with more support tickets might churn more often, but ticket volume could reflect underlying product problems rather than cause departures. A predictive relationship is not automatically an intervention target. Exploratory analysis should generate hypotheses and identify data problems, not convert every visible pattern into a causal explanation.

Reserve evaluation data before using outcomes to guide modelling decisions. For these independently generated snapshots, a stratified random split maintains approximately similar class proportions. Use only the development portion for exploratory summaries and model selection; keep the test portion closed until the approach is fixed. Summary statistics expose scale differences and missing values, while the target mean reports the observed churn fraction. That fraction is a sample estimate, not a universal rate. In real datasets, uncertainty also depends on dependence between observations: repeated measurements from one customer provide less independent information than the same number of distinct customers would provide. ```python from sklearn.model_selection import train_test_split X_dev,X_test,y_dev,y_test=train_test_split( X,y,test_size=0.2,stratify=y,random_state=42 ) print(X_dev.describe()) print(X_dev.isna().mean()) print(y_dev.mean()) ```

Preparation is part of the fitted model, not a separate housekeeping exercise. Median imputation learns a value from observed data, and standardisation learns means and scales. If either transformation uses validation or test rows during fitting, information crosses the evaluation boundary. A scikit-learn Pipeline avoids this by fitting transformations only on the training rows supplied to each fit operation. Here, median imputation handles missing usage, missingness indicators preserve information about absent measurements, and scaling supports regularised logistic regression. Python for data science is especially useful when these steps remain executable, inspectable and reproducible rather than being performed manually in spreadsheets.

The following pipeline produces churn probabilities using logistic regression, a strong starting point for tabular binary classification. Its linear structure applies to log-odds rather than directly to probabilities. Regularisation constrains coefficient magnitudes, helping manage unstable estimates when information is limited or predictors overlap. Numeric preprocessing is sufficient for this synthetic example. With categorical features, use a ColumnTransformer and an appropriate encoder inside the same pipeline, with explicit handling of unseen categories. Keep feature definitions versioned alongside code, and do not assume a fitted coefficient measures a causal effect: interpretation still depends on measurement, omitted variables and the model's structural assumptions. ```python from sklearn.pipeline import Pipeline from sklearn.impute import SimpleImputer from sklearn.preprocessing import StandardScaler from sklearn.linear_model import LogisticRegression model=Pipeline([ ('impute',SimpleImputer(strategy='median',add_indicator=True)), ('scale',StandardScaler()), ('classifier',LogisticRegression(max_iter=1000)) ]) ```

Leakage-safe validation requires both a correctly fitted pipeline and a realistic splitting strategy. Stratified folds suit this independent synthetic dataset, but they are not the default answer for subscription histories. If several rows represent the same customer, consider group-aware splits when assessing generalisation to unseen customers. If deployment predicts future behaviour, use chronological splits and ensure training labels would have been available before each validation scoring date. Overlapping outcome windows may require a gap between periods. A pipeline cannot repair temporal leakage in feature construction or a split that asks an easier question than the system will face in production.

Cross-validation estimates variation across development splits and supports model comparison without consuming the final test set. This example reports ROC AUC, which measures ranking discrimination, and average precision, which summarises precision–recall performance and depends on outcome prevalence. Neither measures business value directly. Compare candidates with a simple baseline and identical splits; retain complexity only when evidence supports it. Fold-to-fold variation is useful diagnostic information, but its standard deviation is not automatically a confidence interval. If extensive tuning is necessary, use a disciplined search and consider nested cross-validation when estimating the performance of the selection procedure rather than one fixed model. ```python from sklearn.model_selection import StratifiedKFold,cross_validate cv=StratifiedKFold(n_splits=5,shuffle=True,random_state=42) scores=cross_validate( model,X_dev,y_dev,cv=cv, scoring={'auc':'roc_auc','ap':'average_precision'} ) print({k:v.mean() for k,v in scores.items() if k.startswith('test_')}) ```

Once the modelling approach is fixed, fit the pipeline on all development data and evaluate the untouched test set once. Report several complementary measures rather than searching for whichever looks strongest. ROC AUC assesses ranking, average precision adds a precision–recall perspective, and Brier loss evaluates squared probability error. Lower Brier loss is better, but it reflects more than calibration alone; inspect a calibration curve when probability accuracy matters. Compare probability metrics with a constant predictor based on development-set prevalence. These generated results demonstrate the evaluation process, not expected performance on real churn data, and should never be presented as such. ```python from sklearn.metrics import ( roc_auc_score,average_precision_score,brier_score_loss ) model.fit(X_dev,y_dev) p=model.predict_proba(X_test)[:,1] print({ 'roc_auc':roc_auc_score(y_test,p), 'average_precision':average_precision_score(y_test,p), 'brier':brier_score_loss(y_test,p) }) ```

A probability becomes an operational decision only after costs, capacity and consequences are specified. A default threshold of 0.5 is not inherently appropriate for retention work. Select any threshold using development data, preferably out-of-fold predictions, and evaluate the locked decision rule on the test set. Examine precision, recall and expected workload at that threshold, including relevant customer segments. High churn risk does not imply high treatment benefit: some customers will leave regardless, while others would stay without an offer. Therefore, multiplying churn probability by customer value does not by itself estimate the incremental return from contacting or discounting a customer.

A/B testing addresses the causal question left unanswered by prediction: does the retention policy improve outcomes compared with the current alternative? Freeze eligibility and the scoring rule before assignment, then randomly allocate eligible customers to an offer or control group. Choose an assignment unit that limits spillovers; linked accounts or households may require cluster randomisation. Pre-specify one primary outcome, its observation window, guardrail metrics and an analysis plan. Retained contribution margin may be more informative than retention alone when discounts have costs. Define the minimum worthwhile effect and calculate sample requirements using plausible baseline behaviour, desired power and the assignment design.

Analyse customers according to their random assignment, whether or not they accept the offer: this intention-to-treat comparison estimates the effect of offering the policy. For binary retention, report the difference in retained proportions with a suitable confidence interval, not just a p-value. Account for clustering where relevant, investigate assignment imbalances and missing outcomes, and avoid repeatedly checking ordinary significance tests until one becomes favourable. Use a planned stopping rule or a valid sequential method. If eligibility includes only high-risk customers, the finding applies to that population; it does not establish the policy's effect on subscribers outside the tested eligibility rules.

Deployment extends the workflow rather than completing it. Store the fitted pipeline, dependency versions, feature definitions, training period and evaluation record together. Monitor input availability, missingness, prediction distributions and operational failures immediately; measure predictive performance and calibration once outcomes mature. Changes in customer mix, pricing or product behaviour can weaken an initially useful model. Feature drift alone does not prove performance deterioration, so combine distribution checks with outcome-based evaluation. Retention interventions also change the data subsequently collected, potentially altering labels and relationships. Preserve experiment assignments and treatment exposure so future analyses can distinguish untreated behaviour from outcomes influenced by previous decisions.

Responsible data science also requires proportionate data collection, appropriate access controls and review of differential impacts across relevant groups. Establish who can approve model changes, what triggers investigation and when the system should fall back to a simpler process. Reproducible notebooks are useful for exploration, but production work benefits from tested functions, data checks and documented decisions. For structured coverage of these connected methods, Erudex's [Data Science: Statistics, Modelling and Experimentation](/courses/data-science) course connects statistical reasoning, modelling and experimentation. Its [practice tests](/practice) can support knowledge checks; consult the course details for the requirements associated with its certificate rather than assuming completion criteria.