# Web scraping app optimizations

**URL:** <https://forums.kodeco.com/t/web-scraping-app-optimizations/144191>\
**Category:** 🙃 Off Topic\
**Created:** [July 14, 2021, 8:09am UTC](https://forums.kodeco.com/t/web-scraping-app-optimizations/144191 "2021-07-14T08:09:24Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![frohmen](https://assets.chunter.kodeco.com/user_avatar/forums.kodeco.com/frohmen/32/104980_2.png) [@frohmen](https://forums.kodeco.com/u/frohmen)\
**Post date:** [July 14, 2021, 8:09am UTC](https://forums.kodeco.com/t/web-scraping-app-optimizations/144191/1 "2021-07-14T08:09:24Z")

</div>

So I’ve been working on an app for quite some tim nowI. t’s actually my first real own iOS app ever, that I wan”to release on the App Stoe

he idea behind the app is quite simple;

It get whats being set what’s being served for lunch today at some local restaurants and displays it to the user by simply scraping it directlyfrom a number of websites .

Up until now the app retrieves what’s being served for lunch today on some local restaurant websites, for now it works like a charm but i’m having a hard time getting my head around that it has to be a more efficient and longterm way of doing this.

What I mean by long term is that if any of the web pages change, it could break the whole app.

One option I’ve been thinking about is to maybe have a PHP script that does the heavy lifting (retrieving, parsing,picking out the right information , but then I would probably have to pay for the expenses of a webhosting solution, which I would rather avoid

So all of this boils down to my question, what do you think I should do?

Alny suggestions is greatly appreciated

---

<div class="post-metadata">

**Author:** ![robertomachorro](https://assets.chunter.kodeco.com/user_avatar/forums.kodeco.com/robertomachorro/32/111603_2.png) [@robertomachorro](https://forums.kodeco.com/u/robertomachorro)\
**Post date:** [July 24, 2021, 3:53pm UTC](https://forums.kodeco.com/t/web-scraping-app-optimizations/144191/2 "2021-07-24T15:53:09Z")

</div>

I think you are on the right thinking path with regards to having the heavy-lifting being done elsewhere. The reason why is UX (user experience) - let me explain: What happens if one of those sites is down? Or unreachable? Or they change the content and you can no longer parse it? The user will be immediately impacted.

If you do the heavy-lifting at your end, upload the results to some service, then the app hits your service, which you control. If something goes wrong with parsing, you will know and fix, but the app doesn’t stop working.

As for the costs, you will be surprised how low they are nowadays - some even free! Do some web-searching and you will find options.

---

<div class="post-metadata">

**Author:** ![frohmen](https://assets.chunter.kodeco.com/user_avatar/forums.kodeco.com/frohmen/32/104980_2.png) [@frohmen](https://forums.kodeco.com/u/frohmen)\
**Post date:** [July 28, 2021, 5:07am UTC](https://forums.kodeco.com/t/web-scraping-app-optimizations/144191/3 "2021-07-28T05:07:59Z")

</div>

Where should I store the file then?  
Can I use Dropbox?

---

<div class="post-metadata">

**Author:** ![robertomachorro](https://assets.chunter.kodeco.com/user_avatar/forums.kodeco.com/robertomachorro/32/111603_2.png) [@robertomachorro](https://forums.kodeco.com/u/robertomachorro)\
**Post date:** [July 28, 2021, 2:22pm UTC](https://forums.kodeco.com/t/web-scraping-app-optimizations/144191/4 "2021-07-28T14:22:20Z")

</div>

I do believe that Dropbox has some kind of public file sharing, I don’t know if I would personally would go that route. I’d rather setup standard web hosting. There are hundreds of solutions out there at very discounted rates and some even are free (Google: free hosting).

If you have some level of experience with web hosting, I would recommend AWS:

> **[Build Free Websites and Webapps](https://aws.amazon.com/free/webapps/)**
>
> Build a website and webapp for free with a AWS Free Tier offer. Check out the free offers and services you need to build, deploy, and manage websites and webapps in the cloud.

---

<div class="post-metadata">

**Author:** ![frohmen](https://assets.chunter.kodeco.com/user_avatar/forums.kodeco.com/frohmen/32/104980_2.png) [@frohmen](https://forums.kodeco.com/u/frohmen)\
**Post date:** [July 29, 2021, 10:26am UTC](https://forums.kodeco.com/t/web-scraping-app-optimizations/144191/5 "2021-07-29T10:26:17Z")

</div>

Is Firebase a good alternative?

---

<div class="post-metadata">

**Author:** ![robertomachorro](https://assets.chunter.kodeco.com/user_avatar/forums.kodeco.com/robertomachorro/32/111603_2.png) [@robertomachorro](https://forums.kodeco.com/u/robertomachorro)\
**Post date:** [July 29, 2021, 2:51pm UTC](https://forums.kodeco.com/t/web-scraping-app-optimizations/144191/6 "2021-07-29T14:51:52Z")

</div>

Firebase is a fantastic framework, and it will definitely solve your issue - in a different way. You can store your data in its database and then pull it in.

Firebase is not a website, which was your original question.

---

<div class="post-metadata">

**Author:** ![frohmen](https://assets.chunter.kodeco.com/user_avatar/forums.kodeco.com/frohmen/32/104980_2.png) [@frohmen](https://forums.kodeco.com/u/frohmen)\
**Post date:** [July 31, 2021, 2:18pm UTC](https://forums.kodeco.com/t/web-scraping-app-optimizations/144191/7 "2021-07-31T14:18:50Z")

</div>

But how and where do I populate the database?

---

<div class="post-metadata">

**Author:** ![robertomachorro](https://assets.chunter.kodeco.com/user_avatar/forums.kodeco.com/robertomachorro/32/111603_2.png) [@robertomachorro](https://forums.kodeco.com/u/robertomachorro)\
**Post date:** [July 31, 2021, 3:13pm UTC](https://forums.kodeco.com/t/web-scraping-app-optimizations/144191/8 "2021-07-31T15:13:52Z")

</div>

I’m afraid that explaining Firebase usage would go beyond the scope of an Off-Topic Forum question. However, I can recommend some tutorials from the [RayWenderlich.com](http://RayWenderlich.com) website:

[https://www.raywenderlich.com/21133526-firebase-tutorial-getting-started](https://www.raywenderlich.com/21133526-firebase-tutorial-getting-started)  
[https://www.raywenderlich.com/21376846-firebase-dynamic-links-getting-started](https://www.raywenderlich.com/21376846-firebase-dynamic-links-getting-started)  
[https://www.raywenderlich.com/20974552-firebase-tutorial-ios-a-b-testing](https://www.raywenderlich.com/20974552-firebase-tutorial-ios-a-b-testing)

RW Pro:  
[https://www.raywenderlich.com/4203-beginning-firebase](https://www.raywenderlich.com/4203-beginning-firebase)  
[https://www.raywenderlich.com/11736578-peter-friese-building-swiftui-apps-with-firebase](https://www.raywenderlich.com/11736578-peter-friese-building-swiftui-apps-with-firebase)

---

<div class="post-metadata">

**Author:** ![frohmen](https://assets.chunter.kodeco.com/user_avatar/forums.kodeco.com/frohmen/32/104980_2.png) [@frohmen](https://forums.kodeco.com/u/frohmen)\
**Post date:** [August 23, 2021, 10:47am UTC](https://forums.kodeco.com/t/web-scraping-app-optimizations/144191/9 "2021-08-23T10:47:33Z")

</div>

Thank you for the uppish answer

---

<div class="post-metadata">

**Author:** ![frohmen](https://assets.chunter.kodeco.com/user_avatar/forums.kodeco.com/frohmen/32/104980_2.png) [@frohmen](https://forums.kodeco.com/u/frohmen)\
**Post date:** [September 9, 2021, 10:33am UTC](https://forums.kodeco.com/t/web-scraping-app-optimizations/144191/10 "2021-09-09T10:33:14Z")

</div>

I’m sorry, but what would I gain by storing it in Firebase?
