TestNet Testing Thread - Test IPFS Integration (Decentralized IP File System)

Trying to figure out how to set up masternode for the first time. Getting close I hope.

Anyway, uploaded 21MB video. Plays right away on Chrome on ipfs.biblepay.org ... ipfs.io was initially spinning but retried after 4 min and the video played finally.

Is the default for Open Attachment ipfs.io or ipfs.biblepay.org ? I'm getting better results with ipfs.biblepay.org on the immediate request.

http://ipfs.biblepay.org:8080/ipfs/QmdMBtPQvuqPyCJgnczh5RcVeS5mi5UJydLzeVvcGcccd4

Thats great!  Yeah, maybe we should make the default open behavior biblepay.org, sounds good.
I had that pausing issue with ipfs.io also - I can imagine they are probably being pounded by tons of 'free' traffic from every direction.

So today Im going to look into adding a fee to the attachment.  For now its just basically a fee per K being attached, and I think we should start by enforcing that it goes to the orphan foundation otherwise the transaction commit fails.  I will also add a process that pins docs that are legitimate and
 have paid PODS fees. 

Jaap, sounds like you are close.
This is a guess, but could you double check the IP and format you typed into the config?  I would first go to your host, and check your public IP address, and make sure it matches.  Also you can type 'ipfs daemon' (after stopping the daemon) and check to see the lines it prints out that it is binding your IP address to other ports; write down the IP that is the public one- see that it matches your host (is your host vultr?) if it does you may just have an error on your gateway entry; it may be case sensitive also, compare the original entry from another node before changing it to the new entry?

One other thing to see: maybe you are already running a service on 8080.  Try to stop the ipfs daemon first, then type 'telnet localhost 8080', see if it replies.  If it does you are running something else on the box, maybe another type of web server.  You would have to stop that other thing first, then start the 'ipfs daemon'.


EDIT:  I just saw your post above that you run the linux rig hot at home; thats probably the issue; if running from the house, you would need to open up the firewall port 8080 and forward it to the linux rig, then it should bind the port.  One side issue with running at home:  check to see if your sanc says ENABLED in our list - the new PODS enforcer requires a sanc to be ENABLED to check its PODS quality.

Hmmm...

This is strange.... I'll just post the entire startup proces here (and delete it later).
The firewall on that rig is disabled and the port was already forwarded to that rig in my router.

When I telnet it (making sure the daemon is off) it says 'telnet: Unable to connect to remote host: Connection refused' so I guess there is nothing else running on that port.

Code: [Select]

jaap@jaap-HP-Pavilion-dm1-Notebook-PC:~/.ipfs$ ipfs daemon
Initializing daemon...
Successfully raised file descriptor limit to 2048.
Swarm listening on /ip4/127.0.0.1/tcp/4001
Swarm listening on /ip4/192.168.1.149/tcp/4001
Swarm listening on /ip4/192.168.1.183/tcp/4001
Swarm listening on /ip6/::1/tcp/4001
Swarm listening on /ip6/fdbe:33bf:560:0:1703:2523:7b47:8758/tcp/4001
Swarm listening on /ip6/fdbe:33bf:560:0:3868:73f5:7c0f:da76/tcp/4001
Swarm listening on /ip6/fdbe:33bf:560:0:4366:95b0:9e9c:d91d/tcp/4001
Swarm listening on /ip6/fdbe:33bf:560:0:8c52:a032:7b4a:f626/tcp/4001
Swarm listening on /ip6/fdbe:33bf:560::25e/tcp/4001
Swarm listening on /ip6/fdbe:33bf:560::595/tcp/4001
Swarm listening on /p2p-circuit/ipfs/QmeUCRZ8edAvqojUZgWBsQ69XaSdWozhwHK39UMrskfYky
Swarm announcing /ip4/127.0.0.1/tcp/4001
Swarm announcing /ip4/192.168.1.149/tcp/4001
Swarm announcing /ip4/192.168.1.183/tcp/4001
Swarm announcing /ip6/::1/tcp/4001
Swarm announcing /ip6/fdbe:33bf:560:0:1703:2523:7b47:8758/tcp/4001
Swarm announcing /ip6/fdbe:33bf:560:0:3868:73f5:7c0f:da76/tcp/4001
Swarm announcing /ip6/fdbe:33bf:560:0:4366:95b0:9e9c:d91d/tcp/4001
Swarm announcing /ip6/fdbe:33bf:560:0:8c52:a032:7b4a:f626/tcp/4001
Swarm announcing /ip6/fdbe:33bf:560::25e/tcp/4001
Swarm announcing /ip6/fdbe:33bf:560::595/tcp/4001
API server listening on /ip4/127.0.0.1/tcp/5001
Error: serveHTTPGateway: manet.Listen(/ip4/84.29.208.33/tcp/8080) failed: listen tcp4 84.29.208.33:8080: bind: cannot assign requested address
Received interrupt signal, shutting down...

The adjusted line in my config is:

Code: [Select]

"Gateway": "/ip4/84.29.208.33/tcp/8080"
The rule on my router is:

Code: [Select]

8080 test
IPv4-TCP, UDP
From any host in wan
Via any router IP at port 8080
IP 192.168.1.149, port 8080 in lan
I triple-checked the external-ip.

Btw, I don't think I had problems running Sanctuaries in testnet this way before. The necessary ports are open, and the external IP almost never changes (only sometimes after a reset of the providers modem). My Sanctuary is:

Code: [Select]

71c44d1cc7e3f29aac6b0c8e9ade3c43cc79da3d0c22f183439d8e440c6a7d3e
I will try and restart the rig. See if that helps.
EDIT: restarting didn't help

Hmmm...

This is strange.... I'll just post the entire startup proces here (and delete it later).
The firewall on that rig is disabled and the port was already forwarded to that rig in my router.

When I telnet it (making sure the daemon is off) it says 'telnet: Unable to connect to remote host: Connection refused' so I guess there is nothing else running on that port.

Code: [Select]

jaap@jaap-HP-Pavilion-dm1-Notebook-PC:~/.ipfs$ ipfs daemon
Initializing daemon...
Successfully raised file descriptor limit to 2048.
Swarm listening on /ip4/127.0.0.1/tcp/4001
Swarm listening on /ip4/192.168.1.149/tcp/4001
Swarm listening on /ip4/192.168.1.183/tcp/4001
Swarm listening on /ip6/::1/tcp/4001
Swarm listening on /ip6/fdbe:33bf:560:0:1703:2523:7b47:8758/tcp/4001
Swarm listening on /ip6/fdbe:33bf:560:0:3868:73f5:7c0f:da76/tcp/4001
Swarm listening on /ip6/fdbe:33bf:560:0:4366:95b0:9e9c:d91d/tcp/4001
Swarm listening on /ip6/fdbe:33bf:560:0:8c52:a032:7b4a:f626/tcp/4001
Swarm listening on /ip6/fdbe:33bf:560::25e/tcp/4001
Swarm listening on /ip6/fdbe:33bf:560::595/tcp/4001
Swarm listening on /p2p-circuit/ipfs/QmeUCRZ8edAvqojUZgWBsQ69XaSdWozhwHK39UMrskfYky
Swarm announcing /ip4/127.0.0.1/tcp/4001
Swarm announcing /ip4/192.168.1.149/tcp/4001
Swarm announcing /ip4/192.168.1.183/tcp/4001
Swarm announcing /ip6/::1/tcp/4001
Swarm announcing /ip6/fdbe:33bf:560:0:1703:2523:7b47:8758/tcp/4001
Swarm announcing /ip6/fdbe:33bf:560:0:3868:73f5:7c0f:da76/tcp/4001
Swarm announcing /ip6/fdbe:33bf:560:0:4366:95b0:9e9c:d91d/tcp/4001
Swarm announcing /ip6/fdbe:33bf:560:0:8c52:a032:7b4a:f626/tcp/4001
Swarm announcing /ip6/fdbe:33bf:560::25e/tcp/4001
Swarm announcing /ip6/fdbe:33bf:560::595/tcp/4001
API server listening on /ip4/127.0.0.1/tcp/5001
Error: serveHTTPGateway: manet.Listen(/ip4/84.29.208.33/tcp/8080) failed: listen tcp4 84.29.208.33:8080: bind: cannot assign requested address
Received interrupt signal, shutting down...

The adjusted line in my config is:

Code: [Select]

"Gateway": "/ip4/84.29.208.33/tcp/8080"
The rule on my router is:

Code: [Select]

8080 test
IPv4-TCP, UDP
From any host in wan
Via any router IP at port 8080
IP 192.168.1.149, port 8080 in lan
I triple-checked the external-ip.

Btw, I don't think I had problems running Sanctuaries in testnet this way before. The necessary ports are open, and the external IP almost never changes (only sometimes after a reset of the providers modem). My Sanctuary is:

Code: [Select]

71c44d1cc7e3f29aac6b0c8e9ade3c43cc79da3d0c22f183439d8e440c6a7d3e
I will try and restart the rig. See if that helps.
EDIT: restarting didn't help

Hi Jaap,

Looking at my vultr node when I bring up ipfs daemon, I see in my case, the line that says "swarm listening on" with the port of 4001:  Mine is listening on the External IP, looks like you have two adapters, and it binds one to one internal ip and another to another internal IP on yours.  Could you try binding 4001 to your external IP (I think you have to edit the config to change that), and also point the 4001 port to that machine in the firewall then restart the daemon and see if it changes the behavior of 8080.

Btw, on your Sanc, does it actually flip over to ENABLED eventually?  Edit:  Your sanc address should start with a "y", I cant see your status on my node.

EDIT 2:  Btw Jaap its really easy to run a sanc side-by-side your running prod sanc - do you have one on vultr?  If you have a lot of problems, it might be easier to do that.

Hi Jaap,

Looking at my vultr node when I bring up ipfs daemon, I see in my case, the line that says "swarm listening on" with the port of 4001:  Mine is listening on the External IP, looks like you have two adapters, and it binds one to one internal ip and another to another internal IP on yours.  Could you try binding 4001 to your external IP (I think you have to edit the config to change that), and also point the 4001 port to that machine in the firewall then restart the daemon and see if it changes the behavior of 8080.

Btw, on your Sanc, does it actually flip over to ENABLED eventually?  Edit:  Your sanc address should start with a "y", I cant see your status on my node.

Thanks for the help Rob!

Yes, I already though it was strange that it was listening to two internal IP's. The other internal IP is my second testnet rig (Win 10 with just a mining wallet), and I actually already tried closing programs on that other rig.

Hmm... Going to look into this a bit later.

About the Sanctuary:
It's actually 'ENABLED' in my own list, but I'm the only enabled sanctuary in my list, two others seem online but they have a 'WATCHDOG EXPIRED' label.

Well... If both the IPFS and the Sanctuary don't work like they're supposed to, then I'll just have to fire up another VPS.

Hmmm... Can't get it to work.

When I use the localhost ip the deamon starts, when I use the external Ip it doesn't...

Hmmm... Can't get it to work.

When I use the localhost ip the deamon starts, when I use the external Ip it doesn't...

So whats your IP Jaap, Id like to see if your sanc is enabled on my node?

So today Im going to look into adding a fee to the attachment.  For now its just basically a fee per K being attached, and I think we should start by enforcing that it goes to the orphan foundation otherwise the transaction commit fails.  I will also add a process that pins docs that are legitimate and  have paid PODS fees. 

I hope you're not realistically thinking $5/mo is appropriate fee because Google Drive is $1.99/mo for 100GB of storage. It is a commodity service so realistically, you should expect pennies to store attachments or not even charge at all. $5/mo is a pipe dream is extremely unrealistic.

Whatever fee you collect and have it go to orphan fund is a good idea. Every little bit helps and hopefully the service will scale over time.

Still having some issues with bigger attachments. Tried a 45MB video but I don't see it in the transaction.

So whats your IP Jaap, Id like to see if your sanc is enabled on my node?

84.29.208.33

I hope you're not realistically thinking $5/mo is appropriate fee because Google Drive is $1.99/mo for 100GB of storage. It is a commodity service so realistically, you should expect pennies to store attachments or not even charge at all. $5/mo is a pipe dream is extremely unrealistic.

Whatever fee you collect and have it go to orphan fund is a good idea. Every little bit helps and hopefully the service will scale over time.

Still having some issues with bigger attachments. Tried a 45MB video but I don't see it in the transaction.

Its not smart to compare apples to oranges; we're renting files on a blockchain, not space on a centralized server.  Not sure why you would consider a decentralized chain less valuable than a centralized company who can delete your account at any time, or decide what news is important for us. 

Nah, it shouldnt necessarily be pennies.  It takes what, 20% of a sanctuary to run ipfs?  With that logic why run ipfs at all.  (I wouldnt participate if I was only getting $1 total per month for hosting ipfs).

Anyway we are storing 250 copies of a file while the cloud servers are using RAID or storing across 20 machines for durability, so we are also doing 10* the work per file.

84.29.208.33

You show as UP on my nodes.  Let me see why I dont show as UP on your node.

You show as UP on my nodes.  Let me see why I dont show as UP on your node.

Im *.*.88.12, and the problem was I was running watchman in PROD mode, update to testnet mode - now I see myself as ENABLED, Jaap do you see me as enabled now?

Its not smart to compare apples to oranges; we're renting files on a blockchain, not space on a centralized server.  Not sure why you would consider a decentralized chain less valuable than a centralized company who can delete your account at any time, or decide what news is important for us.

I was speaking from an average consumer that'd not looking to host files that may be deleted or censored. That seems like an edge case unlikely to happen to 99% of common users. For all intents and purposes, I'm not concerned as consumer how a file is distributed (centralized or decentralized). I just care that the URL works when I need access to it.

Is the marketing plan to encourage uploads that is likely to be censored elsewhere? That seems very risky and controversial. If you're expecting $5/mo premium, then I suppose that has to be the group of people we target? If files are on the ipfs.biblepay.org:8080, does that make files censorship proof? Or biblepay would be ultimate authority on what lives and dies on IPFS?

I was speaking from an average consumer that'd not looking to host files that may be deleted or censored. That seems like an edge case unlikely to happen to 99% of common users. For all intents and purposes, I'm not concerned as consumer how a file is distributed (centralized or decentralized). I just care that the URL works when I need access to it.

Is the marketing plan to encourage uploads that is likely to be censored elsewhere? That seems very risky and controversial. If you're expecting $5/mo premium, then I suppose that has to be the group of people we target? If files are on the ipfs.biblepay.org:8080, does that make files censorship proof? Or biblepay would be ultimate authority on what lives and dies on IPFS?

Why even discuss it if we dont agree on "Im not concerned as a consumer how a file is distributed (centralized or not)".  Because in this case, if you aren't concerned that we are working on blockchain technology, then why discuss it.  As that is a night and day difference.

I feel we are making a mistake to not encrypt the files.  No, they will not be censored or deleted if they are encrypted.  If they are not encrypted, they could be censored, deleted, taken down by government request, etc.

The fee is currently set at .0002 bbp per K of filesize uploaded.  We can potentially lower it after we know the system works properly, but it would be doubtful that it should be lowered so low that it fills up 10 gigs of each of our sanc drives.  But I would lower it incrementally so that we receive some moderate usage by the network.

Why even discuss it if we dont agree on "Im not concerned as a consumer how a file is distributed (centralized or not)".  Because in this case, if you aren't concerned that we are working on blockchain technology, then why discuss it.  As that is a night and day difference.

I feel we are making a mistake to not encrypt the files.  No, they will not be censored or deleted if they are encrypted.  If they are not encrypted, they could be censored, deleted, taken down by government request, etc.

The fee is currently set at .0002 bbp per K of filesize uploaded.  We can potentially lower it after we know the system works properly, but it would be doubtful that it should be lowered so low that it fills up 10 gigs of each of our sanc drives.  But I would lower it incrementally so that we receive some moderate usage by the network.

I feel the point of any feature is to help secure funding orphans long-term. This means daily usage by BBP users which leads to higher BBP prices. You don't get to increasing daily users without targeting more of the user segment. Some will care about the technology, but more will care about the UI and how usable the product is.

We've talked about this before, but if files are not public by default, it removes the usefulness of the feature. Can encryption be optinal?

Can "smart fee" (dynamic pricing) be applied to PoDS as well? If storage capacity is close to full, charge more... if not, charge less. It'd be like charging for parking in a small lot. Cheaper when there's fewer cars, but more expensive as there's limited space. I suppose we could extend it and have long-term parking vs short-term as well...

Im back, upgraded $10/month Ubuntu 16.04 Vultr server to v1.1.4.8b of BiblePay

Steps to edit IPFS config in order to expose your IPFS for Biblepay integration:

cd ~/.ipfs
nano config
scroll down to line 47 (the line reads:  "Gateway:" /ip4/127.0.0.1/tcp/8080)

Edit the line to be:
"Gateway": "/ip4/your_sanc_public_ip/tcp/8080"

Stop the daemon and restart the daemon.

Once ipfs is running again, verify you can pull a file from your sanc publically:

http://yor_sanc_ip:8080/ipfs/QmPVMkWe7976YH22quBotbrDMV9tP4qCz9P5tndveKdeGs/hi.txt

Updated ipfs config with my IP Address

Link works!
http://144.202.69.224:8080/ipfs/QmPVMkWe7976YH22quBotbrDMV9tP4qCz9P5tndveKdeGs/hi.txt

I had to open up port 8080

Code: [Select]

sudo ufw allow 8080/tcp