I’m already hosting pihole, but i know there’s so much great stuff out there! I want to find some useful things that I can get my hands on. Thanks!
Edit: Thanks all! I’ve got a lil homelab setup going now with Pihole, Jellyfin, Paperless ngx, Yacht and YT-DL. Going to be looking into it more tomorrow, this is so much fun!
Self hosting nothing changed my life.
So much free time and less stress once I abandoned self hosting 😅
As far as changed your life, there are not too many that i really love, that made a massive difference to how i do things. But there is one:
Paperless_ngx
ALL of my paper work, receipts, transcripts, tax, shares, council rates. Everything goes in there. We no longer have paper lieing everywhere (well, my wife is another matter, still keeps grocery shopping reciepts…). when i get soimething in the mail, i used the paperless app to “scan” it, upload it, then bin the paper.
An actual life change that i didn’t know i needed.
Is it possible for the scans to be stored as files that are readable should paperless crash and I’m not around to get it up and running, or are files stored as weird non-standard file formats?
edit: looks like scans are saved as pdf’s. Thanks for the insight!
It creates searchable PDFs, so no weird format locked to paperless-ngx
Thanks for the insight!
yeah, and it will order them in a configurable manner, based on dates, tags, people, etc. and as things change in the meta-data of the document, it moves/renames the file to suit.
The files are stored in a directory and you can define the default path with an environment variable ( file-name-handling ). If you need a more fine graint solution you can also use storage paths and select it on file level ( storage-paths ). I’m using syncthing to sync the folder structure to my other devices.
Oh nice, thanks!
Why is this better for you than using a folder structure with a decent naming convention? I’ve tried to get started a couple times, but I just haven’t managed to get what’s better about it. I know i’m missing something, and I feel like if I knew what it is i’d be more likely to out in the work to transition.
well, there are a few things:
- using the app to take photos (in a scan sort of mode, where it trims it to be at right angles), really quick and easy, no matter where i am.
- remote access - i can view all of my documents where ever i am.
- easy & sophisticated search. I have my documents assigned to people (me, wife, child, etc). I also assigned them to things like payslips, tax, shares, legal documents, education docs, receipts, etc. it also helps to automatically tag them to some degree of accuracy
- Automatic dating, it is quite good at picking out the date of the document, as seperate to the upload date. and it is easily updatable if it is wrong
- OCR - the documents content is searchable!
- Ease of tax time. I have some financial year views that make it really easy for me to do my tax (Australia), and i dont need to go hunting for paper that has faded in the heat and is no longer legible.
- folders - the documents are placed in a folder structure of your choosing. if you change the details in the document meta-data, it will move it to the correct place.
so, whilst a folder structure would work. this is SOOO much easier, and provides much more functionality as it is not just storage. it also has WAF!
That was a really clear explanation, thanks. Decent remote scanning would be nice. I guess I just have to wrap my head around tags for some of the niceties to make sense, though I guess i’d be no worse off if I just used folders if that’s an option as well.
you’re welcome.
I tend to use document types more than tags now. note that there are a number of meta-data fields:
- correspondent
- docuemnt type
- tag
i started with tag, but now mostly use a combination of the doc date, type and correspondent. Then use the search bar for specific documents.
Commenting here to save this and also to create engagement.
did you know that you can save a post, by clicking the star?
also, appreciate the engagement :D
I also am creating engagement.
Man that was some solid engagement!
I also am creating engagement.
i thoguht you may be a bot as there are 20-odd replies the same, but my guess is you are using an app which is a bit dicky.
can you delete all the repeats?
Tried, app wasn’t letting me delete either… Funny that it was on this specific post.
And if you try paperless and don’t like it, docspell is another great option.
How is your work flow from scanning to paperless? Does it support some kind of upload folder?
Yeah paperless supports an upload folder. My scanner has an ability to scan to a network drive, so I scan things onto a shared drive on my homelab box, paperless consumes the scanned PDF and places it into the paperless “inbox”.
i dont have a scanner, but do use the email function to get my work payslips.
Cool, that’s really easy. I’ll have to bring that up with my gf. She’s basically hoarding printouts and stuff (she’s a teacher) and this might help her in getting it a bit more organized
Yep, supports upload folder, normal upload in the application and also automatically importing from email based on folder/label
i use the app, it is essentially a photo which is resized/shaped to be a rectangle.
Without even opening I had this in mind.
Yo this shit is awesome. Going to be setting this up asap.
That looks really cool. At the moment I scan everything with OneDrive, and sync it with my NAS. However, it doesn’t have e.g. OCR features, it’s pretty basic. Will have a look, thanks!
definitely try it out. You can auto-ingest from the scanner folder and it will do all the rest of the sorting for you. I go in every few weeks/months and look at the recent documents to sort and fix up any meta-data/sorting.
Thanks. I’ve set it up and imported all my existing scans. Works great.
awesome. i think that the initial install “just works”, then you can start to tweak it. just make sure you mount actual directories, not docker volumes, otherwise you cannto see the files on the disk.
Thanks for the suggestion! I tried to do that and have the files reside on a mount (on my NAS) but that didn’t work, resulted in a “chmod” error. So, instead I’ve created a shell script that runs every night and creates a backup & copies the resulting zip file to my NAS :)
By the way, when using docker volumes, you can see the actual files as well. In my case (RPI4) they are located here: /var/lib/docker/volumes/paperless_media/_data/documents
good to know! thanks
With the right permissions you can get to them. ( i needed root, well started with root)
Saved for futures reference
Honestly Plex/Emby/Jellyfin whichever you prefer is a gamechanger because if you have a large library of content then it just cuts the cord from the subscription services.
I’ve always been happy to pay for them until I went on holiday last January and realised that none of my services were working due to going to a country that was out of the way and the only way to access them was to use a VPN.
So having my own Netflix is a great thing.
Tailscale while doing the above is also really cool
Yep. 100% agree. I have a 175TB server. Sure it was expensive to set up initially, but I have all shows and movies I want, always. From all the different services I would have to subscribe to, I imagine I have recovered my initial outlay and I never have to worry about media being removed from the service or it going out of business.
I have things that aren’t even available if I wanted to subscribe. Best thing you can do for yourself.
No commercials, always high quality. Available anywhere, at any time.
I really hope you have that backed up
He/she probably has all his/her movies backed up in the internet ;)
It just takes a really long time to restore from those backups. And weirdly, they’re scattered all over the place…
Depends on your source and connection but I managed to recover 8tb of movies in a couple weeks.
I have a duplicate server off site that I back it up to
Is it useful without piracy though? It would still be expensive to buy all that media? And usually you can’t even download movies etc that you buy online. Am I missing something?
Quite a bit of what I have on my Jellyfin server is ripped from DVDs and Blu-Rays that I already had.
Makes sense. I didn’t think of copying from disks
Probably an ignorant question but the content you use is pirated right? Should I wonder about legal issues since I would keep it at home and connected to Internet? Protected of course I just don’t see too deep into the issue
If you don’t explicitly set a DNS to allow access from outside the local network, all your stuff is private and confined within your local network. As it is with all, let’s say, wifi stuff that goes on in your home.
Edit. What @notorious said
I think you mean explicitly open the port on your router, but even then that’s not true. Plex by default will proxy your traffic so that even closed off servers can be reached. It is pretty easy to disable remote access in the server settings though.
Swinger parties?
Portainer - For docker containers.
AdGuard Home on 2 separate Raspberry Pi Pico W.
HomeAssistant on its own hardware. Home automation
SearXNG - private search.
Whoogle - private search.
Shaarli - Bookmarks.
youtube-dl - downloading videos.
PaperlessNGX - document storage.
Trilium Notes - notes app
These are the ones I can’t live without. All docker containers running on a NAS.
Exactly a couple of things that we (me and the wife) use really often:
- AdGuard Home is IMHO so much easier to use, although it has been a while since I’ve used Pi-Hole.
- CouchDB for the Obsidian LiveSync plugin
- Immich for a self hosted Google Photos alternative
- Nginx Proxy Manager for exposing all of my services
- Vaultwarden is invaluable for us
While Vaultwarden is great I would not suggest selfhosting your password manager unless you do regular backups. Losing all your password cause your server went down is a great way to ruin your day.
I don’t think that’s true. Even when Bitwarden server is down you can still access your Bitwarden vault, use and export all passwords. You can’t save new passwords but using existing ones should work perfectly fine. So, when your server is down/broken, export your vault, fix server and get new Vaulwarden instance up and import your vault again. Thats it. I still find it safer to selfhost it than getting my passwords leaked.
Nevertheless, are backups crucial. But it is relatively easy with vaultwarden-backup and the free object storage of AWS, Oracle and so on.
It’s very easy to back up and encrypted vault to the cloud. Also all bitwarden clients save your info locally, so you wouldn’t lose your vault unless everything you had logged into it with was destroyed simultaneously.
It’s been a while since I last checked Vaultwarden (back then it was still called bitwarden-rs). If they added an export feature, then that definitely makes things easier. The export feature in the client isn’t enough IMO. Last time I tried it, it didn’t export attachments. So if you for example have your SSH key saved in Bitwarden, well then good luck if you loose access to the vault :P
*ruin your year
I would look at this https://youtu.be/uaixCKTaqY0 in regards to nginx proxy manager. It might not apply to you but worth knowing at least.
Shame NPM is so easy to use compared to Traefik. I just bash my head against the wall if I try to use Traefik for anything but local docker containers. Point it at an external service? I would rather shoot myself
I actually find traefik rather nice to work with. I have a few Middleware chains set up, expose service using labels and add the chains to make sure I get the appropriate settings.
If you only use it with your local containers than sure, I have a similar setup myself. But if I try to break from that prison…
Thanks for teaching me about LiveSync, not being able to sync my notes with mobile without an obsidian account has been annoying, but none of the web based interfaces look at nice or as usable as obsidian. Being able to sync everything between desktops and mobile will be really handy.
Parties.
Home Assistant. It’s a rabbit hole, but it’s great. I’ve got motion enabled lights, thermostats for “dumb” heaters, and I track device usage (tablet, xbox) of my kids.
Home Assistant is nice! Have it integrated with some smart lights and smart plugs. Makes it easy to monitor and control everything locally.
We have it set up in our room so that one widget controls the lights, one controls the fans, one controls the monitors, then there’s a master button that we use to turn off everything that doesn’t need to be always on whenever we leave the room.
Want to play with some fancier stuff with it too, but that alone is incredibly convenient.
Do you have a recommendation for smart plugs and/or bulbs that work well w/Home Assistant and have decent security?
Honestly? I just grabbed the cheapest stuff I could find online, hah. Everything I use is basically a mishmash of whatever was on sale at the time. Home Assistant has worked with every device I’ve hooked up so far, and even when they’re different brands I’ve been able to group them up nicely in Home Assistant’s interface.
Can’t speak to security, unfortunately. While it’s certainly an important concern, my budget has been pretty limited to whatever I can find in multipacks for under $30 ¯\_(ツ)_/¯
I recommend you stay away from anything that requires any kind of portal to setup. If you have to download an app and create an account in order to pair the device or get it setup then don’t use that equipment. There’s a near endless array of sensors and things you can get now that work using Z-Wave, ZigBee, and even HomeKit that work DIRECTLY with HA, meaning that they don’t require commands to be sent up to the cloud via the Internet and then come back down to your device.
Do you have a recommendation for smart plugs and/or bulbs that work well w/Home Assistant and have decent security?
I wish Home assistant was more conducive to running on Kubernetes. I tried it but so much of the local discovery doesn’t work without being in the same LAN as all your IoT devices.
Oh yeah totally agree with you. Got a Google Coral stick the other day after waiting for OK prices and it’s a really nice game changer if you have cameras around the house. Managed to get notifications when my cats are nearby, all through Home Assistant.
Hosting a wedding has a pretty good chance to be life changing
I did this and it led to hosting a baby within my wife. Was pretty steep learning curve and now have zero downtime.
So, if I understand correctly it at least had life changing consequences.
and now have zero downtime.
You mean your service availability is better than five nines??
I have a PiHole, my own EdgeRouter that is behind the Verizon router, a UPS, a wired switch, a SiliconDust HD HomeRun to convert my cable to a stream, my Hue controller, my Camera DVR, and a Pi4 hosting network storage.
It all fits neatly in a 6U closet rack. I use the EdgeRouter to host a VPN I can connect into to manage things for the house, and also use it to dial out to a VPN, so I can connect the TVs in the house to a VPN abroad.
I also have a Smart Garden powered by a raspberry pi, connected to a rain barrel, a water pump, some solenoids, and some moisture sensors.
If you spend some time learning how docker/podman works you’ll be able to host practically anything!
A NAS or Nextcloud or some other way of having files available remotely.
Having a big box with a lot of storage that you can put things on from anywhere is so incredibly useful.
Do you think it’s safe for the average joe to forward ports in the router to access things from the outside?
Have it be accessible over Tailscale (or similar) and that alleviates a lot of the access concerns. No need to setup port forwarding either.
Similar might be running Wireguard yourself, right? Albeit if memory serves that setup tends to require port forwarding, so maybe not (or maybe I set it up wrong).
Tailscale uses the Wireguard protocol (in userspace, not kernel) along with a user and IP management system, a STUN system and a relay so they can provide easy management and connectivity even behind NAT or CGNAT. The relay uses https headers to hide the traffic, which provides a slower connection but allows connectivity in networks that block UDP or VPN traffic.
Installing a Wireguard server would use a kernel implementation of the WG protocol, but you have to open a port on the server side for it, and manually create the peer configuration and public/private keys for them. It is slightly faster, but not as easy to deploy or as versatile when dealing with complicated networks, dual NAT or CGNAT. Also very easy to block on networks as it does not obfuscates the traffic.
I chose to deploy a Wireguard server because it works well for my needs, but if I was behind CGNAT or connected through restrictive networks I would move to Tailscale.
Makes sense!
I set up Wireguard simply to get a rough understanding of how to do so & to try to access some home resources while away, which works well enough across simpler network situations, but as you indicate, breaks down against more complicated network situations.
Port forwarding a wg udp port is way safer than port forwarding some application to login to from the internet. At least with WG you can’t even brute force it or anything, it’s a lightweight protocol that requires a client cert.
Tailscale basically uses NAT hole-punching, doesn’t require any port-forwarding ever, it’s great
Nope. But wireguard works fine and is super easy. I’d recommend something like WG-easy running on the nas. That’s just one port to forward, with a reliable service behind it that does not advertise its presence. That is pretty safe.
Do you have a good tutorial for setting up Wireguard? I was able to setup OpenVPN easily but haven’t been able to get Wireguard to let me access the internet while connected to it. Plus the Android app always says successfully connected even if the server is misconfigured.
I’m using pivpn inside a Debian container in Proxmox instead of using a raspberry pi. You can pick either OpenVPN or WireGuard during install. Mine is running WireGuard with no issues. I feel it’s very easy to use.
2 words: Cloudflare tunnel. Ez-PZ
Cloudflare tunnels are a great alternative to port forwarding
I use Zerotier
Stay away from Plex, if you like to go with Free and Open source.
I’ll start with Jellyfin, and Arr family (sonarr,radarr,prowlarr or Jackett), Vaultwarden and immich
Edit: Learn to spin up docker instances first, as above services would be easier to manage in docker containers and for back ups I prefer Duplicati. And if you run it 24x7 add AdguardHome or PiHole to the mix
Edit1: if you are extremely new to docker instances and find it hard to learn, just spin up CasaOS and you’ll be good to go as it makes spinning up docker containers so easy.
Plex is a far better and user friendly version than jellyfin or emby in my experience especially if you want to share to friends. Granted it’s not open source and has gone commercial route so there is the risk it will continue there. But for now I wouldn’t push to move. If jellyfin can get some more app support and continue to develop and be ready for when Plex messes up then it will take off.
True for users who are already setup with Plex, for them there is no reason to switch as of now, but for a person starting from scratch and setting up things for the first time, it makes a lot of sense to get Jellyfin instead of going Plex. As Plex is moving away from their core of making user’s media available for streaming, and rather focuses in pushing its own streaming content (I know we can toggle that behavior off but it is headache fot new comers, and it should be off by default and if a person likes they can turn on Plex’s streaming content, default should be the user’s content)
A headache? All you need to do is tick a box when you first open the app. There it asks you how you’d like your home screen to look
hey my uncle died pressing that check box
It’s also not fully self hosted.
Only if you want to access it remotely without VPN to your home network. Nothing in Plex forces you to use their servers and you could run it in a network without internet connection
here you are wrong. The very first step in Plex is having a user account not local but on Plex (of course that is going to their servers). So the very 1st step shows it is not fully selfhosted. Neither it is fully private.
if you want to share to friend
Not if they need their own Plex Pass for so many features. The only thing Jellyfin lacks is user self password resets and transcoded downloads. I don’t really see any other advantages in Plex
Transcoded downloads are a pretty big deal unless you want to stream 4K blue ray HDR to your iPhone.
Downloads, not streaming.
Does jellyfin handle audiobooks? For some reason I found the service lacking a couple of years ago, but can’t remember why.
So I got Plex pass and really enjoy it. The Prologue app gives you an audible-like interface for audiobooks that I love. Plexamp for music and Plex Dash to monitor the server. Audnexus matches audiobooks to Audible listings for the metadata. Plex movies and tv match to get metadata, trailers, behind the scenes, cast list, and rotten tomatoes reviews. If Plex ever gets too commercialized/restricted for some reason I’ll switch, but for now I couldn’t be happier.
As far as I know there is basic audiobook support. But I have no clue, because I don’t use it. If I used Audiobooks I wouldn’t be using Jellyfin for them anyway
immich
Did they just nick the Google Photos UI?
Yes its basically selfhosted Google Photos instance kinda thing. There is a great story the Dev shared once, he was paranoid about backing up things to Google or Apple cloud as they have history of sharing it with Feds. So Dev won’t like his family pictures on such platforms, so when him and his partner were to have a baby, he started working on immich, so by the time baby arrives he’ll have a safe platform to backup family pictures.
Wow!! Immich looks great. I’ll be getting that going asap. I actually just started paying for Google drive just to have more space for photos and videos. I’ve always wanted to move over to using my server but I just couldn’t find a great Google photos alternative. This looks perfect.
Glad to know, I was able to help ya avoid that cost. We should be thanking the Dev’s baby, as it helped us all to protect our privacy and our pockets 🤣.
Where do you store your duplicati backups?
I follow the rule of 3 for backups. So I keeps 3 copies of things I like to back up.
- Original (Drive 1)
- Duplicati backups (Sent to drive 2 - Same Machine)
- Using Syncthing I sync The Backup Folder in Drive 2 to a remote Machine
I use borgmatic & rsync.net
Does duplicati have to do periodic full backups?
I’ve used borgbackup / borgmatic. One full backup and only incrementals thereafter.
Okay, so do I did some digging and there is a good news for you. Though it is incremental after 1st go, and there is no way to change it, I checked. Here, these incremental backups are not the traditional incremental backups we know of. They work a little different and no backups are dependent on each here.
For more details checked the answer in forums by lead Dev : https://forum.duplicati.com/t/backup-type-is-incremental/8786/3?u=fedonr
And also here it is by a Contributor who manages their documentation: https://github.com/duplicati/duplicati/issues/3182#issuecomment-382128082
I have only seen incremental from 2nd go, but I’ll check if there is a option to switch that and make it full backups even after 1st go
Where do you store your duplicati backups?
Where are your remote backups? In your secpnd house?
No as I shared I only use a remote machine (which is my old laptop converted to NAS) (2nd house is a dream as of now 🙈)
On a serious note as Duplicati backups can be encrypted, you can use remote Machine, backup to a machine in 2nd house as Syncthing works over relays for remote locations as well, or you can also send encrypted backups to cloud like Gdrive, Dropbox, etc.
Noob here, duplicati is awesome, but I saw some posts about corrupted backups etc so i switched to cmd kopia
This reminds me of my posts on reddit 3 months prior, it all started with Noob here🤣, so even I am a noob. But we all learn bits with time. I read those posts too, but gave it a shot anyway and its been 4 months of using Duplicati, still running without any issues.
I do randomly test it as well, but copy/pasting my stuff and then deleting it from original location, and use Duplicati to restore and works well everytime. I did those tests every 7days for 1st month, but after that it has been 3 months where I do similar tests randomly either 20 days or monthly. And still doing good.
Key part to remember while Duplicati is Versioning, I keep atleast 5 versions of backup (daily backups), and the things I backup are mainly Photos or password manager data. So even if I get a corrupted back up and even lose my system. I’ll still have the 4 other backups which ain’t very old, as its daily backups with 5 versions so, 1 backup per day for last 5 days. So 90% chances are I won’t lose the data, but in case even if I do it would negligible.
Hehe. I cant feel like not noob in this community lol. Honestly I still use duplicaty along with kopia for most critical files and they go to google drive encripted. Kopia does backup of all files to another drive and B2 cloud. Duplicaty is so perfect to use, such a shame someone brought fear in the room 🤣
Noob here, duplicati is awesome, but I saw some posts about corrupted backups etc so i switched to cmd kopia
Would you rate CasaOS over something like ProxMox? I know there is a difference in purpose, since ProxMox is about virtualization and CasaOS is about easy hosting of docker instances.
Do you have an opinion on what is better in the long run for self-hosting?
Is you like to run Multiple OS/VMs on single machine, then Proxmox is your goto, hands down.
CasaOs is more for people like me, who runs a single OS baremetal and like to have multiple docker instances on that same OS. Basically you need a baremetal Debain or supported Linux OS on which you install CasaOS.
CasaOs is more like portainer on steroids, as it offers you Appstore like interface to get one click Docker container installation. But also offers you control (for more advanced users) where if you like you can manager containers and can have terminal/ssh access along with option to change default volume maps set by CasaOS.
One such similar thing to CasaOS is UmbrelOS, please do avoid that, as it only offers one click installations of docker containers with default volume maps (with no way for you to change it) And it lacks all the advanced features to manage containers like in CasaOS. Atleast CasaOs keeps those options hidden away, so once you become a little advanced you can access it.
Also look into Yunohost and Cosmos if you decide to go that route.
I’ll surely checkout Cosmos, as for Yunohost I tried its great but I liked GUI of CasaOS better.
For me nextcloud was the biggest gamechanger. A raspberry pi and a SSD and suddenly I didn’t have to store anything at Google drive anymore. And it’s really beginner friendly, especially when using NextcloudPi
