Analytics models are widely employed in business and run the gamut of applications from data mining and classification models to predictive and prescriptive models. For optimal performance, learn when you should review your model and why it’s important.
The forest industry applies mathematical and statistical models throughout the business. Models are used to analyse and classify forest inventory, develop growth and yield projections, and ultimately schedule tracts for harvest and allocation of wood products to destinations.
All models, regardless of style or flavour, are built to combine input data and logic to solve business problems. Implicit within each model are key assumptions that must be validated prior to deployment. A key part of the model lifecycle is maintenance; changes in business needs and advances in data collection or in modeling capabilities can all be brought to bear against a model to ensure that it remains relevant and continues to provide the business value that it is intended to.
As a maintenance best practice, Remsoft advocate that Woodstock models be reviewed periodically. The periodicity varies depending on a model’s purpose, scope, and frequency of use – for example, a five-year review cycle is likely appropriate for a strategic model that is run every few years but may be too long for an operational model run every month.
Woodstock models are built at a point in time for a specific application using the latest technology and methods available at the time. Once deployed, models tend not to change routinely other than the data inputs, yet the modeling platform and modeling techniques are ever-changing and evolving. At Remsoft we see a broad spectrum of models and continuously investigate and discover ways to improve the efficiency and performance of the Woodstock optimization modeling platform.
A key strength of Woodstock is that it allows users to freely formulate their models; this comes at the cost that a formulation may not be as efficient as it could be, which is assessed and identified when reviewing a model. Periodic reviews ensure that the model is always up-to-date and employs the latest syntax, formulations, techniques, and technologies and thus performs at its best.
An actual example is a model that was integrated with a spreadsheet datamart. At the time of development, spreadsheets were the technology that was supported. A model review identified that the datamart format and template design were impacting performance, particularly the time taken to update the model and load results. The datamart was converted to a relational database format and performance improved significantly.
Another great example is models that employ Mixed Integer Programming formulations (e.g. to force treatment of entire decision units in a single period). Current versions of Woodstock include syntax that declare MIP formulations implicitly making such models easier to use and maintain; MIP models developed more than three years ago are good candidates for review to determine if there is opportunity to leverage this syntax.
Model Fitness for Use
Monitoring model performance is important. Validation of model behaviour and performance extends beyond the development phase; once deployed, businesses typically identify key performance indicators to gauge model performance and track this over time as the model is used. Tracking model outputs vis-a-vis business actuals is key to ensuring that the model continues to represent the business and to properly inform decision making. The emergence of discrepancies is impetus for a model review.
Businesses change and so too do model requirements and business rules (rules may be out-dated or new requirements may arise). A periodic review of model logic, assumptions and business rules is crucial to ensure that the model remains in step with the business and continues to deliver value, and should also reveal opportunities for improvement or change: it may even warn that a model is no longer fit for use and is in need of updating or even redesign.
Changes in business process flows and infrastructure can also impact the data flows between models and information systems. Reviewing and assessing how models interact with information systems may reveal opportunity to leverage functionality in the Woodstock Optimization Studio to automate model updates and streamline the use of models.
Whether you perform model reviews internally or seek third party services such as those offered by Remsoft, we advise that you review your models regularly. Model reviews and performance assessments can identify the need for maintenance, ensuring that models continue to deliver business value efficiently over their lifespan.