lcboapi.com and drinkvine.com

This forum is for discussing current and upcoming LCBO, Beer Store and Grocery Store releases and policies.

Moderators: GregClow, Cass

rejtable
Bar Fly
Posts: 631
Joined: Fri Nov 12, 2010 2:05 pm

Re: lcboapi.com and drinkvine.com

Postby rejtable » Fri Jan 18, 2019 8:34 am

Late chime in here, but thanks for all your work over the years, Ralph. I will really miss having access to this awesome tool.
midlife crisis
Seasoned Drinker
Posts: 1952
Joined: Thu Jan 08, 2004 7:00 pm
Location: Toronto

Re: lcboapi.com and drinkvine.com

Postby midlife crisis » Fri Jan 18, 2019 10:26 pm

Terrible news - I will really miss Drinkvine.
User avatar
Urotsukidoji
Posts: 103
Joined: Thu Dec 15, 2011 2:47 pm
Location: South Of Heaven

Re: lcboapi.com and drinkvine.com

Postby Urotsukidoji » Sat Jan 19, 2019 1:01 pm

it looks to still be updating, something change?
midlife crisis
Seasoned Drinker
Posts: 1952
Joined: Thu Jan 08, 2004 7:00 pm
Location: Toronto

Re: lcboapi.com and drinkvine.com

Postby midlife crisis » Sat Jan 19, 2019 3:54 pm

I'm sure hoping ... Don't tell him! :wink:
User avatar
JaseWescott
Posts: 199
Joined: Sat Jul 29, 2017 4:08 pm
Location: Hamilton

Re: lcboapi.com and drinkvine.com

Postby JaseWescott » Sun Jan 20, 2019 9:56 am

Urotsukidoji wrote:it looks to still be updating, something change?


?

drinkvine will stop working on January 15th, and the site will no longer
be accessible by the end of the month
. Thank you for all of your support
over the past nine years.
DerekD
Posts: 12
Joined: Fri Feb 01, 2019 10:06 am

Re: lcboapi.com and drinkvine.com

Postby DerekD » Fri Feb 01, 2019 12:44 pm

Ralphus - Thank you very much for your had work and dedication on drinkvine. I used your website nearly every day- you provided a very valuable resource that many have come to depend on. Like so many others I'm very sad to see it go. As soon as drinkvine stopped updating I decided to start working on my own LCBO scraper. A few people have asked me to open my scraper to the public, so I've created a very basic website with similar functionality to drinkvine. The site does not rely on the lcboapi.

Ralphus- If I'm stepping on your toes and you do not want my website released please let me know and I'll take it down immediately. It is 100% designed based on your original work. All respect and credit goes to the original drinkvine.com and Ralphus. Without you my blatant knock-off would not exist.

It's a work in progress- there will be bugs/glitches/mistakes. Between family and work I have very little time to work on side projects.

Please take a look at the work-in-progress here.

http://www.caskhunter.com

Known major problems:
- you can't search for stores yet, you can only add them from the big list of all stores
- I used open source shopping cart software to handle logins/accounts. There may be references to "billing information" floating around. I'm removing that stuff as I find it. I only need a postal code.

Thanks,
DerekD
http://www.caskhunter.com
Last edited by DerekD on Fri Feb 01, 2019 9:57 pm, edited 2 times in total.
User avatar
Urotsukidoji
Posts: 103
Joined: Thu Dec 15, 2011 2:47 pm
Location: South Of Heaven

Re: lcboapi.com and drinkvine.com

Postby Urotsukidoji » Fri Feb 01, 2019 1:22 pm

great news on the new site!
User avatar
MatttthewGeorge
Bar Fly
Posts: 754
Joined: Tue Apr 03, 2012 3:45 pm
Location: Woolwich, ON
Contact:

Re: lcboapi.com and drinkvine.com

Postby MatttthewGeorge » Fri Feb 01, 2019 10:12 pm

Love that I see the inventory of lcbo.com. Overall it’s barebones but it works(!), thank you!
untappd/tweetbox/facebitch/snapitychat/etc: @matttthewgeorge
midlife crisis
Seasoned Drinker
Posts: 1952
Joined: Thu Jan 08, 2004 7:00 pm
Location: Toronto

Re: lcboapi.com and drinkvine.com

Postby midlife crisis » Mon Feb 04, 2019 10:02 pm

Derek -- this is great news. Thanks for all the work you've put into it so far. One thing I liked about Drinkvine was that, even though I used it primarily for beer, one could also add the odd whisky or bottle of wine (after an irresistible Belgian review). Any plans to expand beyond beer at any point?
DerekD
Posts: 12
Joined: Fri Feb 01, 2019 10:06 am

Re: lcboapi.com and drinkvine.com

Postby DerekD » Mon Feb 04, 2019 10:45 pm

midlife crisis wrote:Derek -- this is great news. Thanks for all the work you've put into it so far. One thing I liked about Drinkvine was that, even though I used it primarily for beer, one could also add the odd whisky or bottle of wine (after an irresistible Belgian review). Any plans to expand beyond beer at any point?


I knew it was just a matter of time before I got this (very reasonable) request.

Short answer: Yes, I'll give it a shot and see what happens.

Long answer: It's absolutely possible to include all LCBO products. In fact, limiting it to just beer was a lot more difficult than a "full scan" (my original scraper included all products and I had to scale it back).

Here are a few of my concerns:
- A full scan generates a LOT of requests to the LCBO website. My current setup spreads these requests out and (tries to) keep my impact on LCBO.com to a minimum. I want to make sure that I don't cause any problems for their system administrators.
- A full scan also generates a TON of data! When I finished my scraper I let it run once and my database quickly became huge. I'll work on keeping the database efficient, but I don't want it to get sluggish.
- I'm hosting this project on my company's server. I'm not in a position where I can pay for hosting right now, so I have to make sure it doesn't impact their business. I have permission to use their server, but I have to be reasonable...

I should be able to keep my requests to LCBO.com spread out by tracking less popular products and updating them less frequently. If nobody is watching a particular product then I can update inventory weekly, etc. The obvious downside to this would be that fans of Palm Bay Key Lime Cherry may see out-of-date results. Doing most of the scanning late at night will reduce the impact on both the LCBO and my company servers.

My plan is to first improve the functionality of CaskHunter and remove bugs first. (It's only been up for about 5 days) I have a huge list of bugs and features that need to be addressed. Once it's running properly I will switch it back to full scan and see what happens.

Derek
User avatar
Craig
Seasoned Drinker
Posts: 1669
Joined: Wed Apr 10, 2013 9:23 am

Re: lcboapi.com and drinkvine.com

Postby Craig » Tue Feb 05, 2019 7:34 am

How's it implemented? Is it dockerized? Could we just toss it on AWS?
DerekD
Posts: 12
Joined: Fri Feb 01, 2019 10:06 am

Re: lcboapi.com and drinkvine.com

Postby DerekD » Tue Feb 05, 2019 4:15 pm

Craig wrote:How's it implemented? Is it dockerized? Could we just toss it on AWS?


Hi Craig, thanks for the idea.

CaskHunter is a VPS with a LAMP server with an open source shopping cart frontend. The scraper is a combination of PHP, bash scripts, and ImageMagick.

I didn't use docker because I feel that would be overkill for such a small project. I didn't want to spend time setting up docker when I had no plans to migrate and didn't expect scaling to be a concern. Based on comments from Ralph, I don't think we can expect a huge number of users. I spent an hour looking at Amazon ECS to run docker but I can't find a way to estimate pricing, then I gave up and started working on postal code based store searching/sorting.

I like AWS we hosting, but if we went that route there we would need the Lightsail tier which means a monthly fee. Right now I have a pretty sweet deal ($0 / month) that includes a cpanel license. My VPS is limited to 2GB ram and a "2 core" processor. So that's roughly the $20USD monthly package.

Let's keep things as they are for now and re-visit if the site gets sluggish or I get booted off the free server.
User avatar
Craig
Seasoned Drinker
Posts: 1669
Joined: Wed Apr 10, 2013 9:23 am

Re: lcboapi.com and drinkvine.com

Postby Craig » Tue Feb 05, 2019 5:24 pm

I was actually asking just because if it were dockerized you could push updates to a registry (ecr) and let ecs just magically update itself. Plus then it's easy for someone else (me) to pay the hosting.

But free works too, of course.
DerekD
Posts: 12
Joined: Fri Feb 01, 2019 10:06 am

Re: lcboapi.com and drinkvine.com

Postby DerekD » Tue Feb 05, 2019 9:26 pm

Ah, very considerate. I appreciate the offer!

Derek
DerekD
Posts: 12
Joined: Fri Feb 01, 2019 10:06 am

Re: lcboapi.com and drinkvine.com

Postby DerekD » Wed Feb 06, 2019 12:13 am

Minor update: Searching for stores by postal code now seems to be working. Much easier than the long list...

Return to “LCBO, Beer Store and Grocery Stores”

Who is online

Users browsing this forum: No registered users and 7 guests