How can you predict the future, surely no one can do that?

AI, Machine Learning, Big Data are not only IT industry buzz words but are established technologies with real-life applications that have solved several everyday problems in the recent decade. For example, Google Maps gets you from point A to B via an optimal route avoiding high traffic areas using Big Data and Machine Learning. Turn navigation feature was introduced by Google in 2009 and is now widely used by commuters throughout the world.

Fig.1 – Google Maps turn navigation in traffic.

We feel that the real estate industry is lagging in terms of innovation and there are a number of opportunities for application of Machine Learning and Big Data. One of these opportunities is leveraging these technologies for Property Market modelling and forecasts. We are able to model and forecast (Machine Learning) property market trends with a high degree of accuracy thanks to our large dataset (Big Data), which is continuously updated by our crawler, that systematically browses major real estate portals and agent sites in the country and indexes past sales, current listing and rental data.

It is true that there is no “crystal ball” technology to predict the future, but as explained in Google Maps example above, it is possible to model it with a high degree of accuracy. I.e. Google Maps models the traffic for the particular day and hour of your travel, relying on real-time data it receives from the devices of other commuters around you. So, the combination of large volumes of historical data for the traffic patterns seen in recent days/weeks/months as well as its’ real-time inputs allows the Machine Learning algorithm to model the traffic and lay out an optimal route for your particular journey accordingly. Although the semantics of the property market are different from traffic routing, the same principles can be applied.

It is almost impossible to account for all of the factors that influence supply and demand in a particular property market (and in turn drive price growth or decline). Some advisers call for as many as 90 metrics to be taken into account when assessing the future price trends in a particular area. However, the need to rely on these external metrics disappears when Big Data and Machine Learning are thrown into the mix. For instance, Google Maps does not need to take into account the number of lanes, traffic lights or pedestrian crossings along the route. These factors are inherently built into the measurement of speed and frequency of stops collected from thousands of commuters’ devices on previous days and then coupled/analysed with the same data on the hour/day of your travel.

As our model also trains on historical data with new data inputted in a near-real-time manner, it inherently takes all external factors into account when producing the forecast. To continue the analogy, new traffic lanes can be compared to new dwellings, while micro- and macro- economic slumps can be compared to existing and newly introduced traffic lights and so on and so forth. This exemplifies the multitude of parameters whose implications have already been encapsulated in the thousands of recorded past and recent sales in our dataset.

Fig.2 – Sample of HtAG dataset plotted on a map.

It is also important to note that traffic data changes every minute/hour during the day, but similar patterns (cycles) are observed from one day to another along the route of your commute. In the same way, property prices fluctuate quarter to quarter, but evolve in 7 to 10-year cycles, with different peaks and troughs depending on the location. As such, whilst there are multiple external factors that define the amplitude of the cycle in a particular area, there is a high degree of similarity in the pattern of cycles between areas that are in close proximity to each other. For example, council areas in the Greater Sydney region share the same peaks and troughs, however, the Sydney cycle is very different to that of Perth.

Property market data from neighbouring areas (and therefore their cycle parameters) is inputted into our algorithm and is also taken into account when the forecasts are produced. As such, in the same way, that traffic data from nearby roads and streets is taken into account to lay out the route by Google Maps, cyclical patterns of neighbouring areas serve as an input metric providing for a more accurate forecast, while also delineating the overall market dynamic ‘landscape’.

3 thoughts on “How can you predict the future, surely no one can do that?”

  1. The accuracy of the turn navigation feature increases with the rate of adoption by commuters that use it. Do you think as the adoption of such emerging technologies in real-estate increases, their real-life implications will also amplify?

    • Good question, Matt.

      Whether or not the forecasts result in a “self-fulfilling prophecy” will depend on their adoption rate. Whereas almost everyone uses Google Maps, it’s unclear whether the same will happen with data and property transactions. Investors and real-estate professionals largely rely on data before locking in their purchases, however a large portion of transactions are still by owner occupiers who make their decisions based on a “feeling” and are limited only by their budget.

      I suspect that the “sentimental” nature of these transactions is actually the main driving force for the property market cycles. In other words if everyone made decisions with their heads, purely based on data, the cycles may have evolved in a different manner (or completely dissipated i.e. became white noise). That, by the way, would have had an impact on the model’s ability to forecast, given that it takes cycles into account.

      In summary, it is unlikely that the forecasts will create a feedback loop into the property market. That would require mass adoption of data-driven decision making, which is questionable in the owner occupier segment that is the largest buyer group in the market.

  2. I found this discussion very interesting. Alex, I believe that the cyclicity of the property market would still remain even if there is a high adoption of data driven decision making. This is because, cyclicity of the market does not only depend on the sentimentality of investors/buyers but on many other factors such as the availability of capital and economic market conditions. For example, data might show that a particular area will definitely boom but all investors might not be able to ‘flood’ the market due to their budget restrictions. Even if they could, data would take that into account meaning that it might discount the same area due to the resulting overpricing of properties and subsequently highlight a boom elsewhere. This would support the general consensus that Australia is not one property markets, but is comprised of many different submarkets which would engender cyclicity due to the movement of investments and changes in preferences.

    As such, what I found particularly interesting about the self-fulfilling prophecy element of data driven investing which does not eradicate market cyclicity is highlighted in the following section from MacKenzie and Millo (2003). Although lengthy, I think it is essential to quote it in its entirety given preceding discussions:

    “There are few organizational environments as tumultuous as that of the financial derivatives trading. In 1973, the economists Black and Scholes tried to understand how options were priced in the newly instituted Chicago Board Options Exchange. They developed a formula that priced options (i.e., the ability to purchase a particular commodity at a specific price) based on a handful of parameters like its spot price or time to maturity. In the first months of trading, the formula failed to predict option prices, with a typical deviation as large as 30% to 40%. Yet, over time, the formula became accurate to within the low single digits, leading its creators to receive Nobel Prizes. Interestingly enough, however, the formula became accurate only because market participants acted as if it was accurate. Traders used values derived from the formula to inform their bids, the formula became integrated into trading regulations, and assumed in technological infrastructure. The Black-Scholes formula began by modeling option prices. It ended up modeling option prices in response to its modeling of option prices”

    Looking at your service provision page, you claim that your prediction algorithm has also a low digit margin for error for high confidence properties. I can only assume that as the entire economy shifts to a data economy, your model will increase in popularity and precision.

Leave a comment