Price repair fixes and improvement
Fixes:
- fix reconstruction mis-calibration with tiny DataFrames
- fix detecting last-active-trading-interval when NaNs in DataFrame
- redesign mapping 100x signals to ranges:
- no change for signals before last-active-trading-interval
- but for signals after last-active-trading-interval, process in reverse order
Improvements:
- increase max reconstruction depth from 1 to 2. E.g. now 1wk can be repaired with 1h (1wk->1d->1h)
Several improvements to price repair
Repair 100x and split errors:
- Handle stocks that recently suspended - use latest ACTIVE trading as baseline
- Improve error identification:
- Restrict repair to no older than 1 year before oldest split
- To reduce false positives when checking for multiday split errors,
only analyse 'Open' and 'Close' and use average change instead of nearest-to-1
- For weekly intervals reduce threshold to 3x standard deviation (5x was too high),
and for monthly increase to 6x
- For multiday intervas, if errors only detected in 1 column then assume false positive => ignore
Repair missing div-adjust:
- Fix repair of multiday intervals containing dividend
Price reconstruction:
- Move to after repairing 100x and split errors, so calibration works properly
- Fix maximum depth and reduce to 1
- Restrict calibration to 'Open' and 'Close', because 'Low' and 'High' can differ significantly between e.g. 1d and day-of-1h
Miscellaneous:
- Deprecate repair='silent', the logging module handles this
- Improve tests for 100x and split errors
- New test for 'repair missing div adjust'
timestamp of 1900 is older than 100 years,
so yahoo responds with error:
GDEVW: 1d data not available for startTime=-2208994789 and
endTime=1687780922. Only 100 years worth of day granularity data are
allowed to be fetched per request.
this should fix it,
something similar was proposed here:
https://github.com/ranaroussi/yfinance/pull/648
# Please enter the commit message for
your changes. Lines starting
Improve split-repair of multi-day intervals. Because split error can occur within a multi-day interval, e.g. mid-way through week, need to repair each OHLC column separately
Increase robustness of repair 'Adj Close'
Limit price-repair recursion depth to 2
Various important changes to yfinance logging:
Remove handler from YF logger as breaks standard logging practice.
Improve 'download' log message grouping
Move custom DEBUG log formatting into 'yf.enable_debug_mode()', which if called adds the handler. The custom DEBUG log formatting is:
- 'MultiLineFormatter' improves appearance of multi-line messages in any logging mode. Adds leading indent to all lines below first, to align with the first line which is indented by %levelname%.
- Whitespace padding is added to end of %levelname% string up to 8 characters. This resolves different level strings e.g. INFO & DEBUG creating different indents. But this only automatically applies to first line of message - for multi-line messages, 'MultiLineFormatter' extracts amount of padding and applies to all other lines.
- Add leading indent proportional to function depth, because DEBUG generates a lot of messages particularly when repairing price data - same function can be called recursively. Implemented in 'IndentLoggerAdapter', a simple wrapper around the logger object.
- 'log_indent_decorator' inserts 'Entering/Exiting %function%' debug messages, helps organise.
Value Raider