Author Topic: Storage solutions  (Read 220 times)

Offline Krudda

  • Member
  • Posts: 10301
  • 私は 日本語 が 上手 じゃ ありません
    • My Anime List
Storage solutions
« on: February 23, 2017, 11:45:25 pm »
As a few of you guys know, I'm expecting to move in about 2 weeks.
I'd like to set myself up at my new place, so that I can access any one of my ~dozen HDDs/SSDs/SSHDDs from any computer in the house without wires or making a network that requires more than one computer be on at the same time.

Any suggestions?
I plan to eventually start using raid of some kind also.

Online kitamesume

  • Member
  • Posts: 9283
  • Death is pleasure, Living is torment.
Re: Storage solutions
« Reply #1 on: February 23, 2017, 11:47:02 pm »
other than networking the server, how are you supposed to interface with the HDDs? does it simply magically connect to all your devices?

otherwise, you can set up a portable DAS, the DAS can connect via either USB or eSATA.

Offline Krudda

  • Member
  • Posts: 10301
  • 私は 日本語 が 上手 じゃ ありません
    • My Anime List
Re: Storage solutions
« Reply #2 on: February 24, 2017, 12:47:30 am »
That is why I am asking - I don't know how this works.
Years ago I'd just leave a p100 running and map a networked drive on Windows XP (32bit) :P

I'll go google DAS soon.

Offline halfelite

  • Member
  • Posts: 1615
Re: Storage solutions
« Reply #3 on: February 24, 2017, 12:53:41 am »
Couple things.

You cant randomly start using RAID if you dont plan for RAID it will never work.  Second you what operating systems are you comfortable with? As this will decided most likely the direction you go with. At my home I currently run a supermicro SC846 its a 24bay 3.5" server case. It houses an intel I3, with 32GB of ECC ram a cheapo motherboard, a 128GB SSD, and one IBM 1015 SAS card.  My file system is zfs RAIDZ2 running two pools of 12 drives each for 80TB of total usable space. Total cost for the case/mobo/cpu/ram/ssd/sas card was cheap under $500. All the cost was in the drives.

If you are not comfortable to a command line setup like mine or dont need that size. I would look at unraid its a GUI driven and easy to use and can be easily scaled up without much planning as it allows different types of files.

Question to ask yourself for planning
1. how much space do you want?
2. How much money are you willing to spend
3. The size of the unit
4. what is your comfort level
5. Do you want to build it yourself like I did or go with a DIY like unraid or store bought like a synology.

Online kitamesume

  • Member
  • Posts: 9283
  • Death is pleasure, Living is torment.
Re: Storage solutions
« Reply #4 on: February 24, 2017, 02:10:18 am »
speaking of NAS, is there a setup where it could MAC Filter users?
and to make it more convenient, the MAC Filter could apply to select HDDs, some of which isn't filtered at all.

i haven't heard of this type of feature on NASes.
multiple HDDs that aren't nested, having a few HDDs strictly protected and isolated from non-whitelisted guests.

Offline halfelite

  • Member
  • Posts: 1615
Re: Storage solutions
« Reply #5 on: February 24, 2017, 04:13:20 am »
speaking of NAS, is there a setup where it could MAC Filter users?
and to make it more convenient, the MAC Filter could apply to select HDDs, some of which isn't filtered at all.

i haven't heard of this type of feature on NASes.
multiple HDDs that aren't nested, having a few HDDs strictly protected and isolated from non-whitelisted guests.

No. the only way you could even begin to do that is running vlans or IPSEC but even then its a take it or leave it as there is no low level request for a share name for either of them to catch. My network is like so. I have my NAS unit running on my 10GBit side. My kids computers are on VLAN20,Guest WIFI is VLAN30, Adult computers and media players are VLAN10. VLAN10 can access all NAS ports both in/out all day every day. VLAN20 can only access my PLEX port and it is set to only allow between certain hours. VLAN30 can not reach the NAS at all. That is how you do network level segmentation.   

With NFS you can do IP address sharing its not mac filtering but it filters by ip address like so
/share/Media 192.168.0.3(rw,sync)  - this says computer 192.168.0.3 can read/write to the Media Share and no one else
/share/Media2 192.168.0.*(rw,sync) - this says all computers on 192.168.0.? can access Media2

If you are doing Samba/windows shares
Then its all by user name
/share/Media you would give access to say username:admin password: onlyIknow then only people with the username/pass can access
/share/Media2 you would give guest user permission anyone can access with no username/password

its not Correct but you should be able to cheat for NFS
Say you have guest wifi that you dont want to access shares have the guest wifi be assigned ip range 192.168.0.63 or above this assigns IP address then in your NFS do
/share/Media2 192.168.0.1/26 (rw,sync) - this says all computers with ip address 192.168.0.1-192.168.0.62 can access this share
/share/Media 192.168.0.1/24 (rw,sync) - This says all computers with ip address 192.168.0.1-192.168.0.254 can access this share.
« Last Edit: February 24, 2017, 04:33:14 am by halfelite »

Online kitamesume

  • Member
  • Posts: 9283
  • Death is pleasure, Living is torment.
Re: Storage solutions
« Reply #6 on: February 24, 2017, 04:45:39 am »
how about on windows?
i think the file sharing function can permit either globally or just to specific users.
and this user sharing seems to be a type of MAC filtering or am i wrong?

now the question is how secure windows file sharing is.
can a guest bypass it to access files on non-shared or specific-user shared files?

if windows can do this right, then using windows as the NAS's OS should be ok.
although it has the disadvantage of not being able to use ZFS and instead rely on NTFS.


edit: i just remembered something about windows 10, it has a built-in linux subsystem.
does this mean it could directly support ZFS as a non-bootable partition?
« Last Edit: February 24, 2017, 05:26:20 am by kitamesume »

Offline halfelite

  • Member
  • Posts: 1615
Re: Storage solutions
« Reply #7 on: February 24, 2017, 06:29:43 am »
how about on windows?
i think the file sharing function can permit either globally or just to specific users.
and this user sharing seems to be a type of MAC filtering or am i wrong?

now the question is how secure windows file sharing is.
can a guest bypass it to access files on non-shared or specific-user shared files?

if windows can do this right, then using windows as the NAS's OS should be ok.
although it has the disadvantage of not being able to use ZFS and instead rely on NTFS.


edit: i just remembered something about windows 10, it has a built-in linux subsystem.
does this mean it could directly support ZFS as a non-bootable partition?

On windows samba share all you can do is user permission. Its not MAC filtering. Mac Filtering is where you block things by the hardware address.

Windows sharing aka SMB/CIFS is pretty secure its not easy to bypass it can be done with network sniffers and packet spoofing and such but if you have people doing that on your network you will be out of luck for most things.

Windows makes a terrible NAS OS for one reason. Updates every update you have to restart your NAS and its more prone to slowdown just by the nature of Windows OS.  A lot of people run it and are happy with it all depends if you can handle the downtime for reboots.

NTFS and ZFS cant be compared. NTFS functions just fine but as a RAID file system is has a lot of disadvantages. ZFS has a lot of advantages for it and also some disadvantages. ZFS all drives have to be the same size, You can expand a vdev so if you start with 4 drives in raidz1 you can add another drive for 5 drives. You have to create another vdev and add it to your pool. So if you dont buy all your drives at once you have a lot of parity drives on each vdev.

No zfs cant be ran under windows.

I would highly recommend someone looking to start a NAS that does not know ZFS or linux or anything to really take a look at unraid its simple it runs headless and can be controlled from another machine through a web browser.

Online kitamesume

  • Member
  • Posts: 9283
  • Death is pleasure, Living is torment.
Re: Storage solutions
« Reply #8 on: February 24, 2017, 07:44:33 am »
i could shut off the windows update just fine so i don't think i'd encounter issues for that.
on the other hand i find setting up window's built-in file sharing a bit more easier since i'm familiar with it.
but exactly how does window's sharing differentiate between devices? i recall it isn't IP Address because of how dynamic DHCP works.

though with those in mind, one of the main advantages i could think off when setting up a windows-based NAS is that it can double as an HTPC.
strictly speaking if we were to say which HTPC software is best to represent quality then we can't exclude MadVR as a must have.
now combining a small-scale (<10HDDs) windows-based NAS and an HTPC could save a lot of investment and space.


and yes i'm aware of the difference between NTFS and ZFS.
i was simply wondering whether Windows 10's built-in linux subsystem can accommodate the zfsutils-linux module.

Offline halfelite

  • Member
  • Posts: 1615
Re: Storage solutions
« Reply #9 on: February 24, 2017, 07:05:45 pm »
i could shut off the windows update just fine so i don't think i'd encounter issues for that.
on the other hand i find setting up window's built-in file sharing a bit more easier since i'm familiar with it.
but exactly how does window's sharing differentiate between devices? i recall it isn't IP Address because of how dynamic DHCP works.

though with those in mind, one of the main advantages i could think off when setting up a windows-based NAS is that it can double as an HTPC.
strictly speaking if we were to say which HTPC software is best to represent quality then we can't exclude MadVR as a must have.
now combining a small-scale (<10HDDs) windows-based NAS and an HTPC could save a lot of investment and space.


and yes i'm aware of the difference between NTFS and ZFS.
i was simply wondering whether Windows 10's built-in linux subsystem can accommodate the zfsutils-linux module.

Windows sharing does not differentiate between devices there is no hardware level to it. It is all based on NTLM authentication using usernames/passwords. If you setup a share on windows we will call it Movies remove guest username access from sharing tell it on username john can access it. Every machine can see that computer A is sharing a folder Called Movies. But only people that know johns password can access and see what is inside it.  The username that you use for shares does not have to be your login name either you can create a user just for sharing files over the network.

HTPC is an option but 10 spinning drives is a lot louder then you think it is. You would get more savings out of say building a NAS unit you leave in a closet and never see and buying a raspi or something else as your HTPC.

Currently no zfsutils-linux module will not load under windows and for the most part zfsutils-linux is dead as most portions are now in the kernel. There has been talk to get zfs running on windows by the people behind openzfs but it has not happened yet.

Online kitamesume

  • Member
  • Posts: 9283
  • Death is pleasure, Living is torment.
Re: Storage solutions
« Reply #10 on: February 24, 2017, 07:49:16 pm »
Windows sharing does not differentiate between devices there is no hardware level to it. It is all based on NTLM authentication using usernames/passwords. If you setup a share on windows we will call it Movies remove guest username access from sharing tell it on username john can access it. Every machine can see that computer A is sharing a folder Called Movies. But only people that know johns password can access and see what is inside it.  The username that you use for shares does not have to be your login name either you can create a user just for sharing files over the network.

ohh i see, so its an account based system.

HTPC is an option but 10 spinning drives is a lot louder then you think it is. You would get more savings out of say building a NAS unit you leave in a closet and never see and buying a raspi or something else as your HTPC.

only if all of them are spinning simultaneously.
this is the downside of nesting them into one partition either via ZFS, RAID, or JBOD.

Offline Krudda

  • Member
  • Posts: 10301
  • 私は 日本語 が 上手 じゃ ありません
    • My Anime List
Re: Storage solutions
« Reply #11 on: February 26, 2017, 11:31:52 pm »
So after reading the responses, its probably best for me to look into a cheap networked computer.

Hmm. I was hoping to avoid having cat6 cables everywhere (since I cannot drill holes in any walls to lay them under the house), so I guess for now, at least until I buy my own home, I'll keep with my current system of having a few spare portable pockt drives around (2.5" enclosed, USB powered HDDs)

Online kitamesume

  • Member
  • Posts: 9283
  • Death is pleasure, Living is torment.
Re: Storage solutions
« Reply #12 on: February 26, 2017, 11:46:20 pm »
So after reading the responses, its probably best for me to look into a cheap networked computer.

Hmm. I was hoping to avoid having cat6 cables everywhere (since I cannot drill holes in any walls to lay them under the house), so I guess for now, at least until I buy my own home, I'll keep with my current system of having a few spare portable pockt drives around (2.5" enclosed, USB powered HDDs)

wifi dongles works too, its just a tad bit slower than gigabit.

Offline Krudda

  • Member
  • Posts: 10301
  • 私は 日本語 が 上手 じゃ ありません
    • My Anime List
Re: Storage solutions
« Reply #13 on: February 26, 2017, 11:52:40 pm »
Oh true. I already have one that is effectively useless in two weeks. May as well repurpose it.

Offline halfelite

  • Member
  • Posts: 1615
Re: Storage solutions
« Reply #14 on: February 27, 2017, 04:50:51 am »
So after reading the responses, its probably best for me to look into a cheap networked computer.

Hmm. I was hoping to avoid having cat6 cables everywhere (since I cannot drill holes in any walls to lay them under the house), so I guess for now, at least until I buy my own home, I'll keep with my current system of having a few spare portable pockt drives around (2.5" enclosed, USB powered HDDs)

Anything can be cheap.  All the money will be in the drives. I suggest picking up an old intel I3 a cheap motherboard with onboard video and and one pcie 16x port. then 16gb of ram if you find one with 6 sata ports its a good start. Get a small 250gb ssd and load up unraid use the 6 onboard sata ports to start you out. then if you need to expand you can put a sata controller in the pcie slot for more drives.

Offline ridon428

  • Member
  • Posts: 1302
  • Dial Four-Two-Eight Toll Free!
    • Personal Site
Re: Storage solutions
« Reply #15 on: February 27, 2017, 06:00:32 am »
Will a Home NAS work? I think Synology Disk Station would work for you.
I didn't read the wall of text nor the specs sheet of that NAS.

Offline halfelite

  • Member
  • Posts: 1615
Re: Storage solutions
« Reply #16 on: February 27, 2017, 06:33:25 pm »
Will a Home NAS work? I think Synology Disk Station would work for you.
I didn't read the wall of text nor the specs sheet of that NAS.

Most often Synology are to expensive for most people. you are putting down $349.99 on a 4bay. Now with the new 8TB and 10TB drives I think a lot more people could get by on a 4bay system.

Online kitamesume

  • Member
  • Posts: 9283
  • Death is pleasure, Living is torment.
Re: Storage solutions
« Reply #17 on: February 27, 2017, 09:24:21 pm »
yeah, synology is a bit on the expensive side, in fact building your own is a tad bit cheaper.