I am building a document search engine for Indian Listed Equities at thesis-eq.com. Currently, I have added Annual Reports, ConCall Transcripts, Investor Presentation, Quarterly Results & other Exchange Filings for around 1000 companies.
Current Features:-
Text Search
• Across the entire universe.
• Across all companies of a particular sector.
• Date(Month) Filter to look within old reports.
• Different search types to cater to different kinds of queries.
News Feed
• Real time classified news into different sectors
• Create a watchlist with different keywords.
• Search inside news articles.
I am working on following things:
Incorporate image-based PDFs into the database.
Enabling users to save/browse historical searches.
Search inside tables of Annual Reports / Quarterly Results to retrieve a line item number fast.
Summarise ConCall Transcripts using Machine Learning. (Not getting good accuracy as of now)
Increase relevance using semantic search (through synonyms or similar words)
Create an alert for a new document in the saved search. (For example, if any investor appears in a company’s concall)
I would really love to hear feedback from you guys. We can make this platform very powerful if I can understand what kind of queries are useful to investors. Please let me know if you want to see any similar features. I will try my best to include it.
Also if anyone can suggest what resources I can use to build a synonym dictionary, both accounting & sector-specific.
Huge shoutout to Pratysuh Sir for guiding me in difficult times.
This is really cool, I have been dabbling into building something like this as well in my free time as a side project.
How are you training the ML model?
I also thought of building an app to search and listen to conference calls on demand with audio tags for various questions and option to skip past the management financial commentary.
This is really helpful initiative really thankful to you.
I would like to use it for some more time and then wish to give feedback on it.
I had some queries related and would like to know of you are planning to expand it in future
Currently about top 1000 companies are in repository some website were tijori and screener have documents like annual report, investors presentation, etc in it. I really wished of there was option to see those companies which are difficult to find for companies like Black rose, chemcrux or Racl when i was researching these company some months back i found it really difficult/hassle to find relevant news, relevant documents submitted by company if such companies can be added in repository in future will be really great as even tijori and other website have not been able to index them
Properly yet.
2)Can there be option to view documents not on 1 year basis but say 2-3 years combined?
Note: I am not aware about technical challenges which make above suggestion infeasible but it’s just suggestion which i felt will make website better
Hi. I think this is an excellent initiative! All the very best! And thanks for making it easier for investors. I have always been wanting such a tool and finally found one!
Few suggestions:
1)Apart from the company sources, can you try including research reports and credit rating reports as well?
2)Can you please add the concall invites as well, apart from the transcript?
3)Can you try adding the audio recording of concalls too?
Thanks again.
Research Reports - I have thought about it. We can easily do to it. But I am not aware, how can I get the research reports legally. Trendlyne does have reports for many brokers. Kindly let me know if anyone can connect me to analysts/brokers.
Can you please elaborate on the need for ConCall Invite? I thought, it will be noise in database.
Excellent, was doing the groundwork for building similar functionality using Kibana elastic search. Was excited about Tijori, it definitely has some functionality but the scope you mentioned is quite wide.
Saw the portal, my single sentence feedback is “love at 1st sight”
Made single word queries, results response was quick and accurate, company wise format is awesome. Queried multiple words like “higher freight cost” response had 100% relevance, when I changed the query to “freight cost was higher” had zero responses.
Overall, very satisfied as mentioned above.
You are right. Engine is not doing good on semantic search. But have you tried changing search type to “Any of these words” for freight cost was higher rather than default “Exact phrase”. As I tried same query using “Any of these words”, it threw decent output.
I am thinking to make “Any of these words” option default.
“All of the words” gave the desired results. “Any of these words” dilutes the relevance. You gave the flexibility so that users can choose. Very happy with the current version, am sure quality of search will keep increasing as you see more use cases.
By saving keywords, you meant for News or Text Search? If Search, Would you like to save all search parameters like date/ document type or a simple list of keywords only?
Template - Can you elaborate on what kind of templates by basic example?
Watch List - Watchlist for stocks, to get the corporate announcement and news?
Alerts - As I will be having the Feed section, apart from that do want any search alert?
Corporate Action - You will get it in the Dashboard/Feed.
If you don’t mind, I suggest you something to think about it.
I wish to have a website in which I could access the data of those companies which already go broke or unlisted in the past and also discuss about the company.
I am talking about doing some postmortem.
Since yesterday I have been using your portal, quite handy and easy to use. Looks like lot of brainpower went into developing this.
Have some questions and certain inputs, not sure if I want to share here, did not find contact details on the website, looks like you chose privacy option during domain registration.
HI akash this is quite amazing. Trendlyne had a similar feature which would only search over last 2 weeks. This totally does away with any need on my end to subscribe to trendlyne It really helps that the scope of search is broader time wise.
A few feedback:
When I search for “increasing margins” I only see a few results. No way to go to next page for more results. What are the results ordered by? Is there a way we can see all results and also number of results? Surely there are more than 5 companies with increasing margins.
By adding some more metadata on top (sector) one could find trends in sectors too which would be immensely useful. Right now we’d have to manually read hazaaar annual reports or concalls or talk to other people to figure out that China+1 factor is playing out in Chemicals or API companies. all of this can potentially be automated and surfaced to the user directly “here are the top K commonly occuring financial terms in the chemical sector” (margin expansion could be one of them for example).
In the interest of building a robust product consider using any existing enterprise search solutions (i dont know if the economics works out, i dont know if this is a correct use for the search engine’s functionality: this is just a plain dumb user suggestion). Then, the semantic search would come for free. Companies like google have spent decades and billions of $ building semantic search for enterprises. I dont know if it is “affordable” or at what scale a start-up can use it with the decision making financial sense but this is definitely a suggestion worth considering: can you piggyback on someone else’s hard work by paying them a little for it?
Hi, Thanks for writing the feedback. I found it very constructive.
I noticed you are using “Exact Phrase” query which means you are searching when “increasing margins” appears in the same exact fashion. This query is designed for the use case when you want to look for exact same thing as any investor’s name etc. I checked same query with “All of these words”, and got many more results. I will make “All of these words” as default. I have to provide these search types as most users do not know operator queries like “And”, “Or”, “Not” etc.
This is very good suggestion. I will think more in detail to do some text analytics and automatically generate some trends from documents. For China + 1, you can type in “China India” and select conference call as document type with “All of these words”, you will get many companies who are discussing shift.
I am already using some open-source software, without which it wouldn’t have been possible. And I understand my search engine pales in comparison to Google. As far as I know, Semantic is one area where we have to build our own system as everyone’s use case is different. It is all about different kinds of queries users are searching on your platform, and how do you want to tweak algos for different use cases. Taking your example, the query “increasing margins” should also provide results for “margins increased”, “increase in margin”, “higher margins”, “margin increased”, “high margin”, “margin expansion” etc. I am working on these things, hope will start producing better results over time.
Obviously, there is a limitation that it will not throw result by recognizing margins increase from numbers.
@Akash_Agrawal One suggestion would to show common searches which investors can use like margin expansion, moat, operating leverage below the keywords search bar so that new investors will come up and see its power.
It will be good if you can add the search from news feeds as well so that this can become a single source for all searches.
It really helps that the scope of search is broader time wise.