[ art / civ / cult / cyb / diy / drg / feels / layer / lit / λ / q / r / sci / sec / tech / w / zzz ] archive provided by lainchan.jp

lainchan archive - /tech/ - 142



File: 1492725653765.png (74.75 KB, 300x227, ipfs.jpg)

No.142

Huh... where do I start...
What is IPFS?
IPFS(Inter Planetary File System) is a new Peer to Peer distributed web file system protocol that aims to replace the aging HTTP protocol. It aims to be a peer to peer, permanent, hypermedia content addressable file distribution protocol.(Thereby also replacing FTP in many use cases)

Imagine a web where you don't have to pay for hosting your simple static webpage/blog. A web where latency is minimised because instead of downloading from a server on another continent, you can just download it from your neighbor or roommate if he has that content. A web where distributed web apps will continue working even if your ISP is having major breakage.A web that will reverse the process of the centralisation of the internet and make it completely decentralised. A web where content won't be lost so easily by a failure of a single server and links won't break by changing a single letter in the path. A web that you can mount locally on your FUSE and manipulate with the files on it as if they were locally.

In some ways, IPFS is similar to the World Wide Web, but IPFS could be seen as a single BitTorrent swarm, exchanging objects within one Git repository.

In short... IPFS will hopefully become the web 3.0 one day.

NOTE: IPFS is still in ALPHA. At least a quarter of it's features are not even implemented yet.

Cool, but how do I get IPFS to work on my machine?

First you have to download the correct package from:
https://dist.ipfs.io/#go-ipfs

GNU+Linux
1) After you have downloaded the binary, you have to unpack it in it's own directory. And then enter the directory.
2) Now you can install it with:
sudo ./install.sh
Or:
sudo mv go-ipfs/ipfs /usr/local/bin/ipfs
3) Now IPFS is installed. However it needs to be initialised. Initialisation will make a repository at ~/.ipfs.
ipfs init
4) Now you are ready to start the IPFS daemon:
ipfs daemon
5) You can check your connection to the swarm with:
ipfs swarm peers

Troubleshooting:
If when you start the IPFS daemon you get a 'bind: address already in use' error then you need to first learn the PID of the process that is using the deafult IPFS port and then kill it:
sudo netstat -tulpn | grep 8080
sudo kill -9 [PID]

Windows
0)The installation on WIndows is fairly similar as on Linux. However it si slightly different. Well. first you need to unpack the file ofc. Now:
1) Open a command prompt and enter:
echo %path%
Choose the appropriate folder out of those where you will put ipfs.exe
2) Copy ipfs.exe in the chosen folder.
3) Initialise the IPFS repo:
ipfs init
4) Start the IPFS daemon:
ipfs daemon
5) Check your connection to the swarm:
ipfs swarm peers

  No.143

Part 2:

BSD
0) Download and unpack the binary.
1) Now you run the install script:
sudo ./install.sh
1.5) Troubleshooting(If everything is fine you can skip this step):
-Some BSD systems can come preinstalled with another package called IPFS. That package is some NAT package. It might be important so you might want to be careful. But do you want to have IPFS? If so then continue:
First check the output of:
which ipfs
If it says /sbin/ipfs, then:
sudo mv /sbin/ipfs ~/
Maybe you can rerun sudo ./install.sh.
2) Ok now we need to initialise the repo:
ipfs init
3) And now we can start the daemon:
ipfs daemon
4) We can check our connection to the swarm with:
ipfs swarm peers

MAC
OK, I honestly rage quit when trying to install IPFS on MAc OS, as all I had was a VM, and yet I had to somehow enter a failsafe boot which was impossile for me on ym VM, and disable gatekeeper. And gatekeeper is like kicking dead whales down the beach(I wrote this, not the filter). If you still want to try:
0) Disable gatekeeper completely
1) Install IPFS. The BSD instructions can probably help you. Or:
https://ipfs.io/docs/install/#installing-from-a-prebuilt-package

FINAL STEP FOR EVERYONE
-Install the Firefox/Chrome IPFS extensions, so that you can browse IPFS from the browser. I personally use Firefox.

Ok, but what are some interesting things to do? Show me some content and/or distributed webapps?
IPFS based Websites
The IPFS webite:
http://127.0.0.1:8080/ipfs/QmZfSNpHVzTNi9gezLcgq64Wbj1xhwi9wk4AxYyxMZgtCG/
Interplanetary wiki:
http://127.0.0.1:8080/ipfs/QmdDbLyNKCwtBVFhEVQfdBTCnEWmWRJGnm93qa3kzBjVym/#!/home
Gio'd Amelio Blog:
http://127.0.0.1:8080/ipfs/QmdPtC3T7Kcu9iJg6hYzLBWR5XCDcYMY7HV685E3kH3EcS/
Cryptix blog. Also an example of human friendly names on IPFS:
http://127.0.0.1:8080/ipns/www.mindeco.de/
With Love from snow blog:
http://127.0.0.1:8080/ipfs/QmfQiLpdBDbSkb2oySwFHzNucvLkHmGFxgK4oA2BUSwi4t/
Brewster Kahle's blog:
http://127.0.0.1:8080/ipfs/QmavE42xtK1VovJFVTVkCR5Jdf761QWtxmvak9Zx718TVr/
Awesomebrowny's blog:
http://127.0.0.1:8080/ipfs/QmeKZSrzAZ5bhaxH2rvm8HEsiqfKceG1FuLHwwbD3mbnEo/www.awesomebrowny.com/

  No.144

Part 3:

Webapps
IPFSTube:
https://ipfstube.erindachtler.me/
IPFSBin, an IPFS pastebin:
http://127.0.0.1:8080/ipfs/QmTvGbPiS1PaE7AAn4gEszNiYMgdrbMXwLkGnLKYSADs8K/#QmSS3dUE5oKdyDNvpGsYCXgm2mBW4fSDzC5B3NDf7DR8VR
IPFS pics, a distributed web image hosting app(currently down):
https://ipfs.pics
QR code render:
http://127.0.0.1:8080/ipfs/QmccqhJg5wm5kNjAP4k4HrYxoqaXUGNuotDUqfvYBx8jrR/qr#enter%252520text%252520here
IPFS search:
http://ipfs-search.com/

Multimedia
Lot's of recordings from CCC:
http://127.0.0.1:8080/ipfs/QmNwEiWPc8EPc9kGEqdi9UeM5RY8jRJSPVdrtKELFZRH85/
Chaosradio:
http://127.0.0.1:8080/ipfs/QmeF6AutCb1HKzrtpYb7VAECcHgtuodJ7xjDdbPJmcSriE
Risc-V workshop:
http://127.0.0.1:8080/ipfs/QmdycyxM3r8b2pHx3M63Xd8NUfsXoEmBnU8W6PgL9eY9cN
Atari 2600 + IPFS:
http://127.0.0.1:8080/ipfs/QmbKxNNCxBox7Cmv3jiUZbiG3zpzmtnYzVUuKHxfAjvpyH/
Anon's qt website(beware it's an 8ch user, enter on your own responsibility):
http://127.0.0.1:8080/ipfs/QmP7LM9yHgVivJoUs48oqe2bmMbaYccGUcadhq8ptZFpcD/
Stanford IPFS seminar:
http://127.0.0.1:8080/ipfs/QmSd1buim52MtQP1SH2XhN7FC8c1KdFrFKBNmgdUKFdss8/Stanford%20Seminar%20-%20Juan%20Benet%20of%20Protocol%20Labs.webm
Graffen.dk:
http://localhost:8080/ipns/QmTm7RdPXbvdSwKQdjEcbtm4JKv1VebzJR7RDra3DpiWd7/
Another IPFS homepage:
http://localhost:8080/ipns/QmWTyP5FFpykrfocJ14AcQcwnuSdKAnVASWuFbtqCw3RPT/
ANIME REACTION PICS:
http://127.0.0.1:8080/ipfs/QmVbCbGE39kSpgwX33j3s5XiidgFaY8SdE23VBqtbf8rgc
Memes:
http://127.0.0.1:8080/ipfs/QmRa2z3zTL2XHtheAjYnsuSDiyDn4sKktWrywg31KdnMmX

99% of those, but not all were taken from this list:
https://www.reddit.com/r/ipfs/comments/63ev6h/list_of_ipfs_websites/

And a lot more stuff can be found here:
https://github.com/ipfs/awesome-ipfs

Ok, you got me hooked. How can I contribute?

Well you can look here, for fun stuff to do:
https://ipfs.io/docs/examples/
Fun stuff like passing the video hash to your mplayer and letting it play video from IPFS:
https://ipfs.io/ipfs/QmNZiPk974vDsPmQii3YbrMKfi12KTSNM7XMiYyiea4VYZ/example#/ipfs/QmRFTtbyEp3UaT67ByYW299Suw7HKKnWK6NJMdNFzDjYdX/videos/readme.md

To add stuff to IPFS you gotta have the daemon up, and then you can either add stuff trough the firefox extension by going to the webui of the extension. Or you can add stuff via CLI:
ipfs add myfile
You can now read it with:
ipfs cat [hashofmyfile]
You can upload folders with:
ipfs add -r folder

Now to truly learn about IPFS:
https://arxiv.org/pdf/1407.3561.pdf

...Well I think that's it. Anything further... you should better learn on your own.

Please, talk about IPFS and it's accompanying technologies. Also share your own and any interesting hashes you find here.

  No.152

Show should server-side processes work

  No.153

>>152
How*
Body is short :///

  No.154

>>152
That's one of the first things people ask...

Dynamic server side content is done via one or more of the following:
-IPNS-Inter Planetary Name Space-Serves as a name space, can be used with DNS for human readable names.
-IPLD-Inter Planetary Linked Data-Among many other things it does routing inside IPFS.
-Public Key Infrastructure
-distributed agents
-signalling

You can see more on it here:
https://github.com/ipfs/faq/issues/28

  No.165

I hope they'll implement something to be anonymous or to use TOR without leaking your IP through DHT.

  No.176

>>165
It's not one of their goals.
The problem with anonimity is that the internet ever since it's inception was never designed to be anonymous.

As for IPFS, you will be able to connect trough i2p and tor to it.

  No.205

>>142
Could somebody explain the difference between this and Freenet to me?

  No.206

>>205
Freenet works by downloading content on freenet to your disk randomly without your choice. This is why it will never work for businesses as it could bring illegal content onto their machines, including cp. No matter how much the data on the freenet container is encrypted, it is a legal nightmare.
IPFS downloads only the content you requested. Then you seed it until your garbage collector runs trough it. Unless you 'pinned' the content and then you seed it indefinitely.

Also Freesites on freenet are pretty much incapable of dynamic content, while IPFS has already demonstrated capabilities of dynamic content.

  No.207

>>206
So basically, anyone (LE) on the network could potentially see if you downloaded certain content. Great.

  No.208

>>207
which network do you mean?

  No.213

How is content permanent? Will I need to have part of the internet in my hard-drive to seed it?
How will a site not die/run out of seeds?

  No.214

>>206
Thanks for the explanation. What happens once nobody seeds a website any more and nobody has it in their stored files? As far as I am aware, what keeps Freesites up is that they're stored across the Freenet and that basically makes them always reachable, I personally have visited some sites from the early 2000s.

  No.215

>>213
>How is content permanent?
Content isn't 100% permanent. It's 99% permanent.
However it is still miles better than HTTP. Because you can have a server on IPFS except it acts more like a seedbox in a swarm of peers.
In HTTP when a server dies the content is gone.
In IPFS when a seedbox dies there's a good chance that someone else has it. Plus the internet archive has already started using IPFS.

Plus they are also working on Filecoin which is a cryptocurrency where you will mine cryptocoins by hosting rare data on IPFS. You will be able to exchange that for bitcoins. You a mere mortal will be able to make some neetbux basically by browsing the net. And this will keep a lot of the content alive.

>Will I need to have part of the internet in my hard-drive to seed it?

No. And better so, because businesses would never accept for legal reasons that I explained earlier in this thread. Only whatever you download will stay in your repo for a while. It works a lot like a git repo as well. The price of hard drives per gigabyte has beeen dropping faster than bandwidth price anyways.

What is however 100% permanent on ipfs is linking. HTTP uses location addressing which is terrible, because it is enough to change the name of the content or even a single letter in the URL of the content and it will break all of the links leading to that content.
IPFS on the other hand uses superior content addressing. That is as long as the contents of files a,b,c,d,...,z are the same IPFS will treat them as one file, regardless of where those files are or of their name.
I.e. 1) You put a meme onto IPFS and it get's 'link A' by which people can access them.
2) Someone else puts the same exact meme onto IPFS, IPFS notices that they are the same and starts treating it as if it is the same and it also get's 'link A'. A lot like torrents where multiple users seed
3) You decide to remove the file, but it will still be available on the same link because the person from step 2 still has t.

IPFS should primarily be compared to HTTP. Because that's what it aims to replace(not in all use cases, but most of them).

  No.216

>>214
If we're talking worst case scenario. The website dies. But hey, if we're talking worst case scenario a nuclear holocaust could wipe freenet as well.

Anyways in the case of content permanency and especially in the case of link permanency IPFS is a lot better than status quo which is HTTP, without having the drawbacks of freenet.

The internet archive has already expressed interest into IPFS and they will probably archive a load of stuff. Plus there's filecoin that I already mentioned. A cryptocurrency that will incentivize people to host the rare stuff for money.

But here's something interesting, once something dies on IPFS, if someone then finds the dead stuff stashed somewhere on a CD, and then he decides to reupload it to IPFS, all the formerly dead links will work again.

You can even prestage stuff before you actually upload it to IPFS.

Or you can even mount IPFS locally via fuse(if you want a tutorial on that, just ask me). Then you can manipulate with content on IPFS as if it were on your own machine. Then you have basically mounted the (ipfs) internet on your own machine. Which is insane.

IPFS also enhances your capabilities of manipulating data on the internet, as it's also a file system.

  No.229

File: 1492894342834.png (631.93 KB, 93x199, 1469850313741-1.jpg)

by the way I fixed https://ipfs.niles.xyz/ , it was soykafting the bed when it tried to scrape /lam/ which doesn't exist anymore

  No.273

>>215
>However it is still miles better than HTTP. Because you can have a server on IPFS except it acts more like a seedbox in a swarm of peers.
> In HTTP when a server dies the content is gone.
In other words, you don't control the content anymore. Why would any commercial entity want that.

And why would anyone mirror my noobsite for free, except for some hypothetical $just_another_cryptocurrency (aka not free).

I'll just pay a hoster $1 a month, it has less layers of complications inbetween. And when I stop paying, content disappears on both instances. This makes zero sense to me: How does throwing software and complicated handwavy concepts magically make storage cheaper? If it doesn't, why would things be stored longer when nobody pays for it?

  No.274

>>215
>What is however 100% permanent on ipfs is linking. HTTP uses location addressing which is terrible
It is also human-readable if you want it to be. hashes aren't.
are metadata like sensible filenames stripped by ipfs-linking or do they magically auto-restore when downloading an ipfs link?
I assume making sense of my reaction image collection gets much harder.

  No.276

>>142
I tried to understand and appreciate this, but I find myself unable to. This seems just like a throw-together of buzzwords and afterwards try to imagine a problem which it solves.

Imagine in some case in a certain scenario some metrics are better than some other... what is it even? Software? Protocol? Network? Filesystem it is definitively not.

One point I'll give it: It perfects the "cloud" idea without even mentioning it: complete loss of control over what is stored where.

Is there a graph that shows depending on parameters A, B, and C the file is stored N times? Or is it probabilistic: "is stored n times with n of the probability distribution N[A,B,C]"? Does it have some peer distribution mechanism like DHT? Some rsync or bittorrent-style chunking mechanism?

As far as I can read from the documentation, DHT is used for the database of pointers to nodes that possess certain files.

And it uses some bittorrent-style sharing-maximization algorithm: So I can only visit a website at useful download speed when I seed something? Doesn't this punish asymmetrical bandwith subscribers and people who clear their webcache?

It looks like some people put much effort in this, but what do they want to achieve? Somehow everything, but "not exactly".

  No.279

>>276
>what is it even? Software? Protocol? Network? Filesystem it is definitively not.
It's a protocol.
It's a file system not in the sense of a disk file system, but in the sense that it facilitates data storage and access.
>Or is it probabilistic
Yes
>Does it have some peer distribution mechanism like DHT?
Yes it does. S/Kademlia DHT.
>Doesn't this punish asymmetrical bandwith subscribers and people who clear their webcache?
The whole big deal of this is that it is p2p and it distributes the web. If it wouldn't be p2p, there would literally be no point to it.
>asymetrical bandwidth
If it succeeds, ISP's would need to adopt.
>people who clear their webcache
Hey, you wanted to clear your webcache.(IPFS repositories can be set with a space limit if you want to)
>Somehow everything, but "not exactly".
Yes. It's a revolution of the web, not an evolution. Or well, it's more that HTTP was stagnant ever since it's inception. Even HTTP 1.2 was nothing more than a hotfix. IPFS does however want to keep as much API compatibility as it can.

  No.280

>>279
Just to add up to my previous post. If IPFS succeeds, it will be a paradigm shift. And it's showing signs of early adoption even in it's alpha phase.

  No.284

>>273
I agree, I don't understand how this will be any different than Freenet other than being able to decide what you host.
Most of us don't even have our browsers save the cache anyway, why would anyone have incentive to host anything?
>Filecoin
Yea, but that defeats the purpose distribution. There is going to be giant warehouses that specialize in mining Filecoin, making it infeasible for the average person, just like what happened to Bitcoin

  No.285

>>279
>>279
>>Does it have some peer distribution mechanism like DHT?
>Yes it does. S/Kademlia DHT.
maybe I was not specific enough: I saw the DHT part for the data-to-possessing node db. I mean, is there a mechanism for evenly distributing data chunks over the network to maximize availability on higher churn rates? The "at which nodes to store my descriptors" part of a DHT scheme.

  No.286

>>279
>If it succeeds, ISP's would need to adopt.
Just as a remark, the asymmetric bandwidth is more due to the infrastructure not supporting it (long copper cables made for lowfreq communications) iirc.
I however agree with you that forcing telcos to deploy fibre everywhere would be great, only am I not quite sure that will happen.

  No.291

>>276
It's a protocol with an implementation. You can read the paper if you want to know what it actually is and does. The TL;DR is that it's torrents, but seeding one file improves you reputation for leeching any other file. Your reputation is tied to the public private key pair that your client generates on installation. Implementations are expected to keep track of who seeds well and prioritize better seeders. This encourages users to seed others and avoid wiping their identity.

>>206
Conforming IPFS nodes can forward arbitrary data. This is incentivized as a way to build up reputation. I don't know if the current implementation does it, and I don't know much about Freenet, but IPFS seems similar. The following lines from the IPFS paper agree with me.

>In some cases, nodes must work for their blocks. In the case that a node has nothing that its peers want (or nothing at all), it seeks the pieces its peers want, with lower priority than what the node wants itself. This incentivizes nodes to cache and disseminate rare pieces, even if they are not interested in them directly.

If a block has nothing that others want, it can try and search for what the peers want.

>Peer.send_want_list(WantList).

>While the connection is open, nodes advertise their want_list to all connected peers. This is done (a) upon opening the connection, (b) after a randomized periodic timeout, (c) af- ter a change in the want_list and (d) after receiving a new block.
>Upon receiving a want_list, a node stores it. Then, it checks whether it has any of the wanted blocks. If so, it sends them according to the BitSwap Strategy above.
Want lists are saved after being received. This may or may not be temporary.

  No.292

>So basically, anyone (LE) on the network could potentially see if you downloaded certain content. Great.
The want list is a list of block hashes, not files or names. It's possible that different files may overlap blocks, like the header to binary format or a section of transparency on a bitmap. Also, if your node is the kind that does leech on behalf of other nodes (There's nothing barring this and it's incentivized by the protocol.), it cannot be clear which blocks are meant for you and which blocks are meant for others. I'm not sure how the laws where you live would interpret either of those. There is support for public key encryption and every node already has a key pair, so it's possible something like Tor could be created on top of it, but I'm not sure how that would work exactly.

  No.299

>>284
>There is going to be giant warehouses that specialize in mining Filecoin, making it infeasible for the average person, just like what happened to Bitcoin
Doesn't matter. At least it means that files onIPFS will be stored on such datacenters for a really long time.
You will probably still be able to cash out if you are an early adopter.

  No.301

I would be interested to see >>273's question answered:
>And why would anyone mirror my noobsite for free, except for some hypothetical $just_another_cryptocurrency (aka not free).

Choosing what to mirror just makes me think that people will mirror sites they like, but lesser-known sites have a good chance of dropping off the network if the original creator leaves. Is the goal of IPFS "99% permanent for popular content"?

  No.303

>>301
The goal of IPFS is 100% link permanency and 99% permanency for all content.
Eventually probably stuff like the internet archive will find your stuff.
Plus you can have your own seedbox.

As much as I like IPFS, IPFS can not break the laws of physics.

  No.304

>>303
>Eventually probably stuff like the internet archive will find your stuff.
>Plus you can have your own seedbox.
Ok, so lesser-known sites do have a good chance of dropping off the network if the original creator leaves.

If you say IPFS has 99% content permanency because the Internet Archive is involved, then HTTP has 99% content permanency for exactly the same reason.

  No.305

>>304
Not him but I guess the argument would be that if no-one saved it it's simply because it wasn't worth saving. Say you ran a small page with a dedicated community, you'd probably encourage your users to mirror the site in case something happened. If they didn't care enough to do that, it wouldn't be a big loss if it disappeared.

  No.306

>>304
No it doesn't because HTTP doesn't provide them with the required functionality. Or rather, it doesn't make it easy. What about all the other stuff that was mentioned that keeps the content on it? Combine all of that, and then by the time IPFS reaches production quality it's gonna be pretty damn hard to erase content of it. You also know that your statement about HTTP simply is not correct about the way it is now.

Heck I bet that 1000's of people will upload their dickpics to ipfs and they will cry for being unable to remove them.

I repeat, I never said that IPFS bends the laws of physics.

  No.307

>>304
Dude, if you want to use freenet or something, just go and use it.

However, after 17 years it is sure that freenet will never be used by normal people, and is incapable of dynamic content on freesites.

  No.310

>>214
Worst case scenario: literally not one person is seeding/hosting it?
Same thing that happens to every file or website ever in the same situation

  No.311

>>304
>If you say IPFS has 99% content permanency because the Internet Archive is involved, then HTTP has 99% content permanency for exactly the same reason.
Internet Archive requires people to know of its existence and manually check to see if the website they're looking for exists there; with IPFS, IA can just "drop in" as a host for sites that would have otherwise died.