I have been working on a system to automate collection and analysis of business information. I wanted to share where I was at and see if anyone had suggestions.
The proverbial itch that I am scratching is that I need to make a lot of decisions on market and product strategy but I always find it hard to get timely, in-depth information to support my decisions. It's been a challenge to track what my partners are doing, what new products my competitors have released and what trends are emerging in the industry. Conferences are great, calling your friends help, searching the internet is time consuming and frustrating.
I have been building a library that:
1) monitors changes in company websites
2) monitors news from trade magazines
3) scans the Internet for related information.
It then analyzes the information, applying rules and weights to filter and categorize. Then, I have a web front to make it easy to access/modify/comment/share/RSS feeds etc.
The first product I did was for the IP Video Market Info
(the domain I am in). Now I am building the infrastructure to make it easy to replicate it for other business areas. I put a site together for Hawaii Tech Information.
to give you a sense of how it might apply to a local topic.
On the Hawaii side, the idea is that you can get Google alerts or Google news but you get a lot of noise. You can read the Advertiser, the Star Bulleting, PBN, etc. My thinking is that by automatically scanning, filtering, aggregating and cataloging, it's easier to access. You can subscribe to the RSS feed for different topics or companies as well as examine an archived collection online.
I am certainly still experimenting and trying to understand what would best solve real user's problems so any feedback or suggestions would be appreciated.
P.S. - It's all Ruby. The engine is packaged as a Ruby gem and the front-end for the sites are Rails. Relative to C#, which I was doing previously, Ruby and Rails was frustrating at first but now I much prefer it.