I have a simple need: I just wish to be able to programmatically retrieve pricing for a Fund (or Stock) identified by a SEDOL/ISIN.
No other provider I’ve managed to find offers this (to the non-professional investor), and screen-scraping has practical limits (and providers tend to get particularly huffy about it).
Failing this, even a way to simply confirm whether a given SEDOL is available on the freetrade platform would be enough to make me switch!
My use-case - which I’d welcome feedback on - is that I’ve built a home-brew system to gather fund prices and produce a daily email report of the top funds per (Morningstar) sector, the concept being that short-term investing in the funds which are consistently the best performing in well-performing sectors should be a good way to maximise returns.
I used to do this by spidering Trustnet, which was efficient and worked well - but then they moved to client-side JS rendering which would have meant migrating to use something such as Selenium to retrieve any data.
At this point I changed to using the ft.com Morningstar data - but many of these funds do not appear to be available from UK brokers. Given the lack of any available APIs to date I tried a highly rate-limited simple HEAD against HL’s public website… and got my IP banned for my efforts . YouInvest, on the other hand, has a much more sensible website that’s behind a CDN - but they’ve also raised questions with my ISP simply based on the volume of requests (although I’d argue surely not the impact - it’s at most a single HEAD per fund per day, and only for new SEDOL/ISINs which haven’t been previously encountered!). I had some success making the request via a public HTTP(s) proxy… but I should probably just migrate the whole setup to a cloud platform and get a Lambda/Function to populate the database.
@finki I don’t want to cause any harm to your service - what rate-limiting do you impose? I was planning simply to use the ‘ukSymbol’ call to confirm the existence of the given ISIN on freetrade/Trading 212 platforms… is this a reasonable plan? Is this the lightest-weight method of determining ISIN validity/availability?
T212 has an exposed API you can query against easily by ISIN. Or scrape their page that displays the asset universe.
FT produce a Google sheet… or you can hit the Costs and Charges HTML page as the URL is a simple ISIN parameter.
Aren’t they easier routes?
No rate limits currently but it surely is coming soon. I’ve hit so many million monthly queries it is getting out of control and performance is suffering.
If you’re using the “uk” Symbol it suggests UK stocks only… in which case call the LSE exposed API and convert ISIN to symbol/tidm?
All those options probably faster than FinKi for your use case.
I also just looked at the HL and YouInvest sites and I can see queryable APIs you can hit with Sedols and ISINs and reverse engineer and the youinvest site uses a Morningstar feed that you can also manipulate to query by ISIN… so you could probably revert to using Morningstar if you used that previously !? Or if you wanna go back to using trustNet then go through something like ScrapeStack to render JS and make it scrapable
Don’t get me wrong … use Finki. But your use case seems easier and better served elsewhere. That said I don’t truly understand exactly what you’re trying to do.
I kinda feel I can build something for you… so ping me more details if you’re so inclined and I’ll have a hack when I get time
I’m not a (frontend… or other!) developer so I could have missed something here… but HL has an external API? I actually contacted them and they explicitly denied that such a thing existed… could you point me at any details?
Yes. Have a look at the ‘network’ activity that runs through your browser. Usually XHR , Sometimes ‘ Fetch’… just use your browsers inspector… Crtl Shift I. if you’re using Chrome for example
I had assumed that, should such a thing exist, it’d be predicated on a session-specific token or similar to prevent use by anything other than their own website (… without additional effort, at least)!
I’ll take a look… (although it’d probably get my IP blocked again, knowing my luck).
They seem quite vociferous in the belief that their (extensive…) T&Cs apply to any and every client connecting to their website
(In terms of XHR requests, there only seems to be a clickTrack response with any data)
I note that later in the buy/sell process, HL does begin to predicate requests on SEDOL codes, so I assume they’re chosen to obfuscate these on the majority of their public-facing pages as some means to maintain a competitive advantage…
I think I noticed the share search on HL was driven by Sedol or could be hacked by throwing a singular letter at it … maybe … I looked for 3 seconds. Looks like II use sedol ‘openly’ if you’re interested!
I imagine they limit sedol use due to horrible lse licensing requirements. Whereas ISIN is ‘generally’ less stringent