As you all know in my last post, I suggested a workflow that would go as follows:
- nptech tagger tags a URL
- tagger goes to nptech pligg site
- in a few minutes, they see their tag show up
- they add more details on the pligg site
- Rinse and repeat
I figured I would use Yahoo! Pipes for my suggested Pligg RSS feed import but in order to do this we’d have to clean up the current RSS feed. We can ensure this by creating a separate for-Pligg-only Feed using Yahoo! Pipes.
Why? Well, we need to de-dupe URLs. We also need to make sure the RSS feed only contains the freshest links so that the Pligg doesn’t repeat URLs. Remember, the nptech RSS feed is only loosely coupled and Feedburner always keeps the old items in the feed. The Pligg site cannot tell the Feedburner feed to stop giving it URLs it has already received from the last import. This means that our Pligg has to be extremely careful about taking in items from the nptech feed. I was hoping to use Yahoo! Pipes to add that logic for us.
Unfortunately, Yahoo! Pipes doesn’t have even the simpler tools that programmers need to have in order to make it work for nptech taggers. I was hoping to use Yahoo! Pipes to always present only the last five minutes of RSS data so that the Pligg could just do imports every five minutes from the Yahoo Pipes feed. Simply put, you cannot sort an RSS feed by date and have it enter our Pligg properly. Until this happens, we can’t set up a good workflow for nptech taggers.
We can still use the Pligg to import straight from the nptech RSS feed but this means the occasional hiccup. You’ll see occasional repeats of old URLs for at least the first few days and we’ll never be completely sure of eliminating duplicated URLs. Can we live with this? Please comment below.
Loading ...
Hi there,
Could you let us know what functionality is missing that prevents you from using Pipes the way you want? We do offer sort-by-date on feeds, though there are currently no semantics for saying “later than N minutes ago”. We’d be glad to help, though, particularly if you can point to a specific pipe for the sake of debugging.
Thanks for trying out Pipes!
Yes, sort-by-date is offered if you sort on pubdate. However, we need that “later than N minutes ago” semantics. That means a way to say “current date and time” and I’d prefer a way to keep UTC (I forget the RFC for it right now) for pubdate as well.
Maybe what you suggested could be implemented using http://soarack.blogspot.com/