Stefan, any part of the 3rd addition elaborates how to effectively setup a security master for equities or any persistent reference part of the data infrastructure. 2nd addition only had a brief mention, I believe.
Depends on what your starting point is. We discuss many of the the relevant techniques in chapter 2-4 https://www.ml4trading.io/chapters/. In my experience, security masters are best sourced from data providers directly; if they don’t have one, it’s a bad sign. Therefore, the practical challenges are often more about combining such references from different sources, which we discuss. Feel free to message me directly here or on LinkedIn if you have more specific questions.
Thank you, Stefan! A practical problem I have, after spending some time developing my own “data infrastructure” is having a dataset of precomputed factor values ready to be picked up by a backtester. That includes features I developed during my 20 years as a fundamental guy at a US macro hedge fund. For the US I do use Quandl. For non US equities (outside of Bloomberg and Refinitiv) FMP is a good candidate as an “anchor” provider. The data model is different, of course, and blending the two datasets is not ideal. So the question is how to approximate data management fluency working with fundamentals without rebuilding the database and changing the data model. You are making it real easy in the 3rd addition for non-fundamental data.
For fundamentals (and US equities prices), Sharadar is useful (via Nasdaq Data link, which also incorporates Quandl since the acquisition, or direct), it comes with security master. Matching symbols across datasets over longer periods can be a challenge with unknown unknowns; depending on your plans it might be a smoother path than trying to merge the data.
The conclusion I am coming to is to rebuild the fundamental setup I have. Sharadar / Quandl are about 80% there in terms of necessary fields and structure. For example, notion of other currencies is completely missing, so is concept of exchange/holiday calendars. FMP has the additional fields, but lacks the hardness for certain concepts, like corporate actions, beyond splits and dividends. Then again, I suspect the data quality and coverage could be less than desired. Bottom line, I am leaning towards blending the semantics with certain compromises accepted. New stuff can be bolt-ons or mapped additions. Problem is it won’t be a small effort. Would rather be testing and trading :)))
Stefan, any part of the 3rd addition elaborates how to effectively setup a security master for equities or any persistent reference part of the data infrastructure. 2nd addition only had a brief mention, I believe.
Depends on what your starting point is. We discuss many of the the relevant techniques in chapter 2-4 https://www.ml4trading.io/chapters/. In my experience, security masters are best sourced from data providers directly; if they don’t have one, it’s a bad sign. Therefore, the practical challenges are often more about combining such references from different sources, which we discuss. Feel free to message me directly here or on LinkedIn if you have more specific questions.
Thank you, Stefan! A practical problem I have, after spending some time developing my own “data infrastructure” is having a dataset of precomputed factor values ready to be picked up by a backtester. That includes features I developed during my 20 years as a fundamental guy at a US macro hedge fund. For the US I do use Quandl. For non US equities (outside of Bloomberg and Refinitiv) FMP is a good candidate as an “anchor” provider. The data model is different, of course, and blending the two datasets is not ideal. So the question is how to approximate data management fluency working with fundamentals without rebuilding the database and changing the data model. You are making it real easy in the 3rd addition for non-fundamental data.
For fundamentals (and US equities prices), Sharadar is useful (via Nasdaq Data link, which also incorporates Quandl since the acquisition, or direct), it comes with security master. Matching symbols across datasets over longer periods can be a challenge with unknown unknowns; depending on your plans it might be a smoother path than trying to merge the data.
The conclusion I am coming to is to rebuild the fundamental setup I have. Sharadar / Quandl are about 80% there in terms of necessary fields and structure. For example, notion of other currencies is completely missing, so is concept of exchange/holiday calendars. FMP has the additional fields, but lacks the hardness for certain concepts, like corporate actions, beyond splits and dividends. Then again, I suspect the data quality and coverage could be less than desired. Bottom line, I am leaning towards blending the semantics with certain compromises accepted. New stuff can be bolt-ons or mapped additions. Problem is it won’t be a small effort. Would rather be testing and trading :)))
Looking forward to reading the book…
The “Volatility: Realized, Implied, and Why It Clusters”. link directs to a "page not found"
Sorry, try again.
No worries, just wanted to give ya a heads up!
Really enjoying all the extra content, can’t wait for the new edition