Just started self hosting this instance. Nothing on the docs mentioned anything about storage considerations.

  • RuudM
    link
    fedilink
    English
    22410 months ago

    This is lemmy.world after 4 weeks:

    58G	pictrs
    34G	postgres
    
    • @i_lost_my_bagelOPA
      link
      fedilink
      English
      6510 months ago

      Considering this is going to be around a 5 user instance at most I think I’ll be good for awhile. Thanks!

      • manitcor
        link
        fedilink
        English
        47
        edit-2
        10 months ago

        im running 50 users right now, subbed to A LOT of communities, seeing db growth of about 100mb per day.

          • terribleplan
            link
            fedilink
            English
            57
            edit-2
            10 months ago

            It’s all about how many communities your user(s) subscribe to since your instance basically acts as a mirror for those.

            My instance has been running for 23 days, and I am pretty much the only active local user:

            7.3G    pictrs
            5.3G    postgres
            

            edit: I may have a slight Reddit Lemmy problem

            • @Pleonasm@programming.dev
              link
              fedilink
              English
              810 months ago

              So if you’re the only user (let’s assume for ease) then, that represents all the updates (posts, comments, votes) from each community that you are subscribed to?

              • terribleplan
                link
                fedilink
                English
                1610 months ago

                Yeah, and I purposely subscribe to (or sometimes have a dedicated “federation helper bot” account I run subscribe to) most of the most popular communities on the most popular instances so I can get a decent sampling of what’s going on in the fediverse on the “All” feed. So I assume my storage usage is maybe a bit higher than what an “average” single-user instance may be…

                • @bdonvr@thelemmy.club
                  link
                  fedilink
                  English
                  410 months ago

                  lmao same here. I have a spare account that I use to sub to everything worth subbing to. I haven’t automated it yet though.

                • @ipkpjersi@lemmy.one
                  link
                  fedilink
                  English
                  310 months ago

                  Ooh, that’s a really good idea, I need a federation helper bot/account when I start self-hosting a Lemmy instance!

            • Alexander Kehr
              link
              fedilink
              English
              310 months ago

              Do you also post stuff? I mean my instance is only about an hour old, but I’ve subscribed to some communities, yet I don’t see the picture service consuming the S3 storage I’ve configured

              • terribleplan
                link
                fedilink
                English
                1210 months ago

                Lemmy caches every thumbnail of every post for like a month or something using Pictrs, so that storage will eventually hit a sort of equilibrium and start growing much more slowly (only reflecting post/thumbnail volume during the cache time).

                Between profile images, community banners/icons, post images etc. there are probably a few dozen images that will be sticking around for the long haul at the moment.

              • @bdonvr@thelemmy.club
                link
                fedilink
                English
                510 months ago

                Your instance only caches thumbnails, so it won’t take much space. The full images are served from the remote instance. So you basically only store whatever your users upload.

      • Dran
        link
        fedilink
        English
        1610 months ago

        Question if you know: does a lemmy instance have to be publically accessable to work? Like, if I make an instance on my homelab can the instance “fetch” content and serve it faster locally? Could I reply to a post and have others see it? Etc

    • @HappyHam@lemm.ee
      link
      fedilink
      English
      1810 months ago

      Now I wonder how viable it would be to support video hosting. The answer is almost certainly “God no!”

      • GatoB
        link
        fedilink
        English
        110 months ago

        It is viable through other hostings

      • @BigWigglyStyle@lemmynsfw.com
        link
        fedilink
        English
        1510 months ago

        At the end of the day the vast majority of what needs to be saved is text. If media content is embedded, the the server just has to save the path to the file not the file itself.

    • lightrush
      link
      fedilink
      English
      5
      edit-2
      10 months ago

      Feels like this will benefit from some sort of fuzzy deduplication in the pictrs storage. I bet there are a lot of similar pics in there. E.g. if one pic or a gif is very similar to another, say just different quality or size, or compression, it should keep only one copy. It might already do this for the same files uploaded by different people as those can be compared trivially via hashing, but I doubt it does similarity based deduplication.

  • Netto Hikari
    link
    fedilink
    English
    5010 months ago

    This is my small instance with way fewer users than lemmy.world.

    11G	pictrs
    5.2G	postgres
    
    • @Molecular0079@lemmy.world
      link
      fedilink
      English
      910 months ago

      Out of curiosity, how long has your instance been up? Just want to get a sense of how fast storage is increasing for you.

    • Kushan
      link
      fedilink
      English
      810 months ago

      How has your Lemmy experience been on a self hosted instance? I’m currently using lemmy.world and it’s very error prone, would self hosting reduce those errors at the expense of anything? Does federation take long or do you find you’re getting federated content quickly enough?

      • Netto Hikari
        link
        fedilink
        English
        1310 months ago

        The experience has been pretty good, to be honest. No instability, easy updates, etc. I find federated content quite quickly, because I use this script to populate the “All” feed.

      • @bdonvr@thelemmy.club
        link
        fedilink
        English
        710 months ago

        You won’t get any old content, so that’s a downside. You’ll only get content after you start federating. Unless someone votes or comments on old content.

        Other than that the only downside is spending time maintaining and updating it.

    • @holycrap@lemmy.world
      link
      fedilink
      English
      3010 months ago

      I really hope it doesn’t get purged if lemmy is to be a Reddit replacement. A lot of the value Reddit had was obscure knowledge and making google searches actually usable.

      • @Molecular0079@lemmy.world
        link
        fedilink
        English
        2110 months ago

        I think as long as the original community the post is in doesn’t purge the data, it’s fine for other instances to purge if necessary.

        • @punio7@lemmy.world
          link
          fedilink
          English
          310 months ago

          Exactly, when dealing with big data, you need a strategy to archive old data. You can’t just store everything in one DB. Smaller instances may not feel like keeping all the date from all the time. Even big instances should have a mechanism to move old data do different databases.

      • Kayn
        link
        fedilink
        English
        410 months ago

        Are you planning on donating to instances that don’t purge old data?

  • Tugg
    link
    fedilink
    English
    1410 months ago

    Small instance with about 3 users and myself online for about 2 weeks.

    pictrs   930M
    postgres 1.4G
    
  • key
    link
    fedilink
    English
    13
    edit-2
    10 months ago

    Depends. If you have a lot of users posting a lot of pictures and you use pictrs out of the box config, then a lot. If you are just running a few users with finite communities being synced then a lot less. The number is going to vary a lot as lemmy grows and gets older so hard to document realistic expectations. But docker images are probably going to take up more disk space than actual contents for now unless you get quite big. My instance is just me and I just threw my PG volume into a tgz to move servers and it’s less than a gig after a month ish.

    • @bdonvr@thelemmy.club
      link
      fedilink
      English
      5
      edit-2
      10 months ago

      The lemmy.world admin said above that their instance currently takes up less than 100GB

      Though this will accrue over time I suppose.

      But for self hosting? You should be good for a long long time. The only pictures stored are the ones you upload, the rest is just text.

  • hitagi (ani.social)
    link
    fedilink
    English
    910 months ago

    My instance eats up almost 100MB everyday. It mostly depends on what your users subscribe to. It was barely growing on my first few days until I invited a couple of friends over to try it out.

  • @i_lost_my_bagelOPA
    link
    fedilink
    English
    810 months ago

    lol lemmy died almost immediately after i posted this time to figure out what the hell caused that

  • Kayn
    link
    fedilink
    English
    7
    edit-2
    10 months ago

    My instance dormi.zone has been running for around 3½ weeks now, has a 3-digit amount of users and hosts a community with little more than 1000 subscribers. Here’s how much storage it currently takes up:

    • 4.9 GiB pictrs
    • 6.2 GiB postgres

    In the default Ansible configuration, storage will mostly be accumulated by log files that are automatically generated by Docker and deleted whenever you restart the Docker containers.

  • Philip
    link
    fedilink
    English
    710 months ago

    After hosting my own instance with just me for ca. 2 weeks:

    1.99Gi pictrs

    5.21Gi postgres

  • @holycrap@lemmy.world
    link
    fedilink
    English
    610 months ago

    Unless they changed all of the comment and post ids to bigints that’ll probably bring the site down before it runs out of storage. In defense of the lemmy developers they have been receptive to feedback, so I don’t think it’ll take long for that to be fixed if it hasn’t already.

  • @GustavoM@lemmy.world
    link
    fedilink
    English
    2
    edit-2
    10 months ago

    I haven’t tried it out just yet, but I’d say… a whole lot. Depending of how popular your instance is, but… your PC will be hammered in any way.

  • MrJukes
    link
    fedilink
    English
    19 months ago

    It depends on how many communities you end up pulling in. Your instance will only sync with communities that a user on your instance is subscribed to.

    • MrJukes
      link
      fedilink
      English
      1
      edit-2
      9 months ago

      I’ve had my instance running for about 1 week and I’m the only user.

      2.1G pictrs

      2.5G postgres