I have a simple need: I just wish to be able to programmatically retrieve pricing for a Fund (or Stock) identified by a SEDOL/ISIN.
No other provider Iāve managed to find offers this (to the non-professional investor), and screen-scraping has practical limits (and providers tend to get particularly huffy about it).
Failing this, even a way to simply confirm whether a given SEDOL is available on the freetrade platform would be enough to make me switch!
My use-case - which Iād welcome feedback on - is that Iāve built a home-brew system to gather fund prices and produce a daily email report of the top funds per (Morningstar) sector, the concept being that short-term investing in the funds which are consistently the best performing in well-performing sectors should be a good way to maximise returns.
I used to do this by spidering Trustnet, which was efficient and worked well - but then they moved to client-side JS rendering which would have meant migrating to use something such as Selenium to retrieve any data.
At this point I changed to using the ft.com Morningstar data - but many of these funds do not appear to be available from UK brokers. Given the lack of any available APIs to date I tried a highly rate-limited simple HEAD against HLās public websiteā¦ and got my IP banned for my efforts . YouInvest, on the other hand, has a much more sensible website thatās behind a CDN - but theyāve also raised questions with my ISP simply based on the volume of requests (although Iād argue surely not the impact - itās at most a single HEAD per fund per day, and only for new SEDOL/ISINs which havenāt been previously encountered!). I had some success making the request via a public HTTP(s) proxyā¦ but I should probably just migrate the whole setup to a cloud platform and get a Lambda/Function to populate the database.
@finki I donāt want to cause any harm to your service - what rate-limiting do you impose? I was planning simply to use the āukSymbolā call to confirm the existence of the given ISIN on freetrade/Trading 212 platformsā¦ is this a reasonable plan? Is this the lightest-weight method of determining ISIN validity/availability?
T212 has an exposed API you can query against easily by ISIN. Or scrape their page that displays the asset universe.
FT produce a Google sheetā¦ or you can hit the Costs and Charges HTML page as the URL is a simple ISIN parameter.
Arenāt they easier routes?
No rate limits currently but it surely is coming soon. Iāve hit so many million monthly queries it is getting out of control and performance is suffering.
If youāre using the āukā Symbol it suggests UK stocks onlyā¦ in which case call the LSE exposed API and convert ISIN to symbol/tidm?
All those options probably faster than FinKi for your use case.
Or I made (but donāt maintain) a very slow broker seach that sounds like what youāre looking forā¦ feed it an ISINā¦ itāll tell you who trades itā¦
I also just looked at the HL and YouInvest sites and I can see queryable APIs you can hit with Sedols and ISINs and reverse engineer and the youinvest site uses a Morningstar feed that you can also manipulate to query by ISINā¦ so you could probably revert to using Morningstar if you used that previously !? Or if you wanna go back to using trustNet then go through something like ScrapeStack to render JS and make it scrapable
Donāt get me wrong ā¦ use Finki. But your use case seems easier and better served elsewhere. That said I donāt truly understand exactly what youāre trying to do.
I kinda feel I can build something for youā¦ so ping me more details if youāre so inclined and Iāll have a hack when I get time
Iām not a (frontendā¦ or other!) developer so I could have missed something hereā¦ but HL has an external API? I actually contacted them and they explicitly denied that such a thing existedā¦ could you point me at any details?
Yes. Have a look at the ānetworkā activity that runs through your browser. Usually XHR , Sometimes ā Fetchāā¦ just use your browsers inspectorā¦ Crtl Shift I. if youāre using Chrome for example
I had assumed that, should such a thing exist, itād be predicated on a session-specific token or similar to prevent use by anything other than their own website (ā¦ without additional effort, at least)!
Iāll take a lookā¦ (although itād probably get my IP blocked again, knowing my luck).
They seem quite vociferous in the belief that their (extensiveā¦) T&Cs apply to any and every client connecting to their website
(In terms of XHR requests, there only seems to be a clickTrack response with any data)
I note that later in the buy/sell process, HL does begin to predicate requests on SEDOL codes, so I assume theyāre chosen to obfuscate these on the majority of their public-facing pages as some means to maintain a competitive advantageā¦
Or I made (but donāt maintain) a very slow broker seach that sounds like what youāre looking forā¦ feed it an ISINā¦ itāll tell you who trades itā¦
I think I noticed the share search on HL was driven by Sedol or could be hacked by throwing a singular letter at it ā¦ maybe ā¦ I looked for 3 seconds. Looks like II use sedol āopenlyā if youāre interested!
I imagine they limit sedol use due to horrible lse licensing requirements. Whereas ISIN is āgenerallyā less stringent