Bible Pay

Read 93230 times

  • Rob Andrews
  • Administrator

    • 4090


    • 97
    • June 05, 2017, 08:09:04 PM
    • Patmos, Island Of
    more
Re: TestNet Testing Thread - Test IPFS Integration (Decentralized IP File System)
« Reply #45 on: September 01, 2018, 09:45:50 AM »
macOS wallet worked fine. Uploaded an image without having to make any adjusts to the file name in the form field.


Good article on using IPFS to share documents where it is only available to the recipient (encrypted with their public key) as one use case:
https://medium.com/@mycoralhealth/learn-to-securely-share-files-on-the-blockchain-with-ipfs-219ee47df54c


Hey all, with our recent drop in price I believe we need a huge end of September release.

Please keep eyes peeled for the next testnet version.
Im working on merging in a spork to give us the ability to test PODC with NO TEAM requirement, and we really need a few cpids to verify this works properly.
We also need to test the blacklist spork, to ensure if a team is changed to a blacklisted team, the payment stops in the superblock.

I'm adding in IPFS integration now, to provide proof-of-document-storage (I suppose we can call this PODS).  Ill explain the spork settings later today.  But in PODS, the next version will provide Durability for the data (meaning for the life of the lease, you can download the file from the gateway).  It will also enforce sanctuary payment by requiring Sancs to host the files that are paid for in the chain (otherwise the sanc loses the sanctuary payment).

Regarding the encryption method, yes, I like that idea for one-to-one transfers, and Im on the fence as to if we should Require that when we go live with PODS, or allow the docs to be downloaded as we have them now (through the gateway).  I understand both sides of the argument- usability for future decentralized websites vs privacy.  The IPFS system is storing the files as blocks, so currently the data is not encrypted.  But we should consider encrypting everything before we go to prod with the feature.



« Last Edit: September 01, 2018, 08:53:43 PM by Rob A. »


  • jaapgvk
  • Hero Member

    • 558


    • 31
    • September 01, 2017, 08:02:57 PM
    • Netherlands
    more
Re: TestNet Testing Thread - Test IPFS Integration (Decentralized IP File System)
« Reply #46 on: September 01, 2018, 12:30:42 PM »
Hi guys,

Bit late to the party, but didn't have physical access to my computers, so I couldn't start my dedicated testnet-rig. Fired up two testnet wallets now. One on Win 10 and one on Xubuntu 16.04.

The Win 10 one is synced and I successfully sent a file to the ipfs:

http://ipfs.biblepay.org:8080/ipfs/QmfCcpUP5oM6L96NPeeQ54SUzXyA7E2wWC2scy5LNL4wV1

The Xubuntu-one has some trouble connecting to peers (already did the 'addnodes') and also seems to have a lot of tBBP although it's a brand new Xubuntu installation and the wallet says it's only synched about 400 blocks thus far.

I haven't started a Sanctuary yet (still waiting for the Xubuntu-wallet to sync), but I did install the IPFS without errors (it seems).

I also started a teamless boinc-account that is building up RAC as we speak :)

One question: my Sanctuaries are hosted on a low-budget VPS, with I think 20GB HDD-space. Can it be possible that Sanctuaries become 'corrupt' (don't know the right phrasing here) when people start sending very large files?


  • Rob Andrews
  • Administrator

    • 4090


    • 97
    • June 05, 2017, 08:09:04 PM
    • Patmos, Island Of
    more
Re: TestNet Testing Thread - Test IPFS Integration (Decentralized IP File System)
« Reply #47 on: September 02, 2018, 07:54:28 AM »
Just to update everyone waiting for the release:  It should be ready within a couple hours.  It's running a little late due to some of the debugging required for ipfs.



  • Rob Andrews
  • Administrator

    • 4090


    • 97
    • June 05, 2017, 08:09:04 PM
    • Patmos, Island Of
    more
Re: TestNet Testing Thread - Test IPFS Integration (Decentralized IP File System)
« Reply #48 on: September 02, 2018, 08:18:50 AM »
There seems to be an indication that there's a max storage you can configure:
https://github.com/ipfs/go-ipfs/pull/4388/commits/0d2cc04fa7691f04b899e0a6dede77db96356e68


For BBP & IPFS, you'd have to guarantee at least 6 months which is the whole point of attaching files within BBP right? If BBP checks that all documents are available to peer on each sanctuary, then these documents need to be "pinned" since garbage collection jeopardize document availability on a specific sanctuary.

Since we don't know the total storage required, some sanctuaries will likely perform the heavy lifting if they have the storage required. Other sanctuaries would likely make a best effort and let garbage collection remove the less accessed documents. I don't know what Rob is thinking about Proof of Document Storage (PoDS), but I could what percentage of the documents you can store on your sanctuary affecting how you are paid.

Since my sanctuary with 50GB storage can only store 50% of the required documents, I'm only eligible for 50% of the PoDS reward. But my other sanctuary with 100GB storage can host 100% of the required documents, is eligible for 100% PoDS reward. PoDS reward could be 10% (e.g. 3.85% of the monthly payouts) of the 38.5% Sanctuary. Initially, every sanctuary would get 100% PoDS reward, but if PoDS takes off, those with more storage can receive the full 100% payment over time.

Note that some of these things can be made into network adjustable sporks, so we can adapt to them.
On the comfortable hosting size per node, I personally think allocating 10-20 gig of storage should be enough, and Jaap, you can set a limit in the config file of ipfs so it does not overrun your drive.  I was thinking, BBP should be relatively cheap per K of storage, but we can probably charge quite a bit for a bigger file.  If its going to be hosted on 270 nodes, and available to the world, that file might be $5 per month.  We can discuss pricing later.  We also have to discuss how the sancs receive compensation for the attachment (currently I have it set as *penalization* if they dont participate in doc storage) but if there is a 1000 bbp fee on a transaction, right now it would go to the orphanage foundation.  This has to be well thought out and implemented for prod.

As far as Sanc penalization, heres how it is set currently:  We do have a spork that turns the PODS system on or off (podsmode=0 means its off in prod).  When it is on, we have a health check feature.  When a sanctuary is assessing proof-of-service, this is the activity of a current sanctuary looping through all known biblepay sancs, and checking if they are online (end-to-end verification), when the sanc assesses the state of another node to be healthy it stores a value of either "ENABLED" or "WATCHDOG_EXPIRED", enabled means the other sanc is online.  So with PODS, we add in an extra similar health check for IPFS.  If the foreign sanctuarys gateway port (8080) cannot serve the healthcheck.dat file properly, the sanc gets marked as "DOWN".  This would actually cause my sanc to not vote on the down sanc for regular sanc payment.    Meaning masternode winners votes would shrink - specifically with the downed sancs not being voted for payment.

However if the sanc is UP, if we have PODS in STRICT mode, it will then try to pull down a file that is paid for and leased in biblepay.  If that file is pulled successfully by hash (we check the actual downloaded files bytes length and the web servers stated byte length also) then we add that file to our own repo if it was missing, and we mark it as downloaded (this is so we dont keep asking for the same failed file), but if that sanc is not able to deliver that particular file, we mark that sanc as "DOWN".

So in this way all sancs who are up and host all the files are fully paid, sancs who are up and only host half the files should technically receive about 50% of the winning blocks still, because they will still receive some UP votes from other sancs who happen to be checking the files that *are* present.  So another words, I think we might want to KISS, and allow payment to be :
SANC REWARD * PERCENT OF FILES SHARED = SANC REWARD * PERCENT OF VOTES = THEORETICAL EQUAL PAYMENT
In practice, if this does not work, we will need to add an affinity of file hashes to votes, but I really would rather resist adding complexity if possible.
(This is how it is programmed now - sancs vote for you if you have the file the sanc is checking for at the time it is ready to vote and you are up).  Sancs do keep a list of what files were checked in memory and loop through different files automatically.

In this next version each sanc has to run ipfs, and in the ipfs config file (see ipfs stats for the location of config) you must edit the config and change the "127.0.0.1:8080" binding to "your_sanc_ip:8080" - this is so you can host a public file gateway.  Note that by default your IPFS does not run a gateway, so this has to be manually changed by everyone.  I have not investigated automating ipfs installs yet etc.

Please try to edit your ipfs configs and restart the daemon, and see if you all can pull a file from:

http://yor_sanc_ip:8080/ipfs/testhash

Once you get that working your sanc is ready to be ipfs enabled.



  • Rob Andrews
  • Administrator

    • 4090


    • 97
    • June 05, 2017, 08:09:04 PM
    • Patmos, Island Of
    more
Re: TestNet Testing Thread - Test IPFS Integration (Decentralized IP File System)
« Reply #49 on: September 02, 2018, 09:19:59 AM »
Steps to edit IPFS config in order to expose your IPFS for Biblepay integration:

cd ~/.ipfs
nano config
scroll down to line 47 (the line reads:  "Gateway:" /ip4/127.0.0.1/tcp/8080)

Edit the line to be:
"Gateway": "/ip4/your_sanc_public_ip/tcp/8080"

Stop the daemon and restart the daemon.

Once ipfs is running again, verify you can pull a file from your sanc publically:

http://yor_sanc_ip:8080/ipfs/QmPVMkWe7976YH22quBotbrDMV9tP4qCz9P5tndveKdeGs/hi.txt

« Last Edit: September 02, 2018, 09:55:10 AM by Rob A. »


  • Rob Andrews
  • Administrator

    • 4090


    • 97
    • June 05, 2017, 08:09:04 PM
    • Patmos, Island Of
    more
Re: TestNet Testing Thread - Test IPFS Integration (Decentralized IP File System)
« Reply #50 on: September 02, 2018, 10:38:11 AM »
I just pushed 1.1.4.8, I'll explain some more after church.

Building Windows now.



  • jaapgvk
  • Hero Member

    • 558


    • 31
    • September 01, 2017, 08:02:57 PM
    • Netherlands
    more
Re: TestNet Testing Thread - Test IPFS Integration (Decentralized IP File System)
« Reply #51 on: September 02, 2018, 01:15:30 PM »
cd ~/.ipfs

Hmm... This is strange...

The IPFS seems to be installed without error, and also running, but this directory doesn't seem to exist.


  • Rob Andrews
  • Administrator

    • 4090


    • 97
    • June 05, 2017, 08:09:04 PM
    • Patmos, Island Of
    more
Re: TestNet Testing Thread - Test IPFS Integration (Decentralized IP File System)
« Reply #52 on: September 02, 2018, 05:51:10 PM »
Hmm... This is strange...

The IPFS seems to be installed without error, and also running, but this directory doesn't seem to exist.
Hi Jaap, please try
ipfs stats repo

The Repo path should be in there.
Then cd to it and nano config.


  • Rob Andrews
  • Administrator

    • 4090


    • 97
    • June 05, 2017, 08:09:04 PM
    • Patmos, Island Of
    more
Re: TestNet Testing Thread - Test IPFS Integration (Decentralized IP File System)
« Reply #53 on: September 02, 2018, 05:58:04 PM »
Windows should be out there now.  1.1.4.8.

Ill set up a sanc and then attempt to edit the OP with additional tests, etc.



  • Rob Andrews
  • Administrator

    • 4090


    • 97
    • June 05, 2017, 08:09:04 PM
    • Patmos, Island Of
    more
Re: TestNet Testing Thread - Test IPFS Integration (Decentralized IP File System)
« Reply #54 on: September 02, 2018, 06:33:14 PM »
Should I blacklist Gridcoin for the blacklist test?  (Rosetta - team 12575), the way this works is we add 12575 to the team blacklist.  (We support multiple teams, so we can also blacklist 30513 (Gridcoin World Community Grid)?

EDIT:  Ok, in testnet I blacklisted WCG Gridcoin and Rosetta Gridcoin.
And I just removed the team requirement for WCG and Rosetta.



« Last Edit: September 02, 2018, 06:41:26 PM by Rob A. »


  • Rob Andrews
  • Administrator

    • 4090


    • 97
    • June 05, 2017, 08:09:04 PM
    • Patmos, Island Of
    more
Re: TestNet Testing Thread - Test IPFS Integration (Decentralized IP File System)
« Reply #55 on: September 02, 2018, 10:59:53 PM »
So far so good, at this point all please upgrade to 1.1.4.8, set up a sanc, verify we are synced, and lets regroup when the sancs say "ENABLED" (we must have some enabled to test ipfs).

I just brought two up, one is enabled now and one is enabling.

The sporks are ready so for now we can use Togo's account as our non biblepay team guinea pig, my CPID as a biblepay guinea pig and we still need a volunteer to set the team as Gridcoin to test the blacklist.

Please set up your IPFS gateway and test with a file from the public IP (as shown in the example above) and then we will regroup.




  • jaapgvk
  • Hero Member

    • 558


    • 31
    • September 01, 2017, 08:02:57 PM
    • Netherlands
    more
Re: TestNet Testing Thread - Test IPFS Integration (Decentralized IP File System)
« Reply #56 on: September 03, 2018, 01:08:35 AM »
Hi Jaap, please try
ipfs stats repo

The Repo path should be in there.
Then cd to it and nano config.

Thanks! I looked into it and the path is correct now. I adjusted the config and tried to start the daemon. Got this error message:

Code: [Select]
Error: serveHTTPGateway: manet.Listen(/ip4/MYIP/tcp/8080) failed: listen tcp4 MYIP:8080: bind: cannot assign requested address
Also, I have joined team Gridcoin (teamid=12575) and have connected my wallet to Rosetta@home :)


« Last Edit: September 03, 2018, 02:06:45 AM by jaapgvk »


  • sunk818
  • Global Moderator

    • 521


    • 36
    • April 24, 2018, 02:02:20 PM
    more
Re: TestNet Testing Thread - Test IPFS Integration (Decentralized IP File System)
« Reply #57 on: September 03, 2018, 02:00:40 AM »
Trying to figure out how to set up masternode for the first time. Getting close I hope.

Anyway, uploaded 21MB video. Plays right away on Chrome on ipfs.biblepay.org ... ipfs.io was initially spinning but retried after 4 min and the video played finally.

Is the default for Open Attachment ipfs.io or ipfs.biblepay.org ? I'm getting better results with ipfs.biblepay.org on the immediate request.

http://ipfs.biblepay.org:8080/ipfs/QmdMBtPQvuqPyCJgnczh5RcVeS5mi5UJydLzeVvcGcccd4

Quote
Status: 0/unconfirmed, broadcast through 3 nodes
Date: 9/2/2018 23:58
Source:IPFS-AttachmentTo: Donate to Orphans yadZnJ3hD3FRC8CiLZEVNqejvQFgNtu5ci
Debit: -10 000.00000000 tBiblepay
Transaction fee: -0.00100000 tBiblepay
Net amount: -10 000.00100000 tBiblepay

IPFS Document: https://ipfs.io/ipfs/QmdMBtPQvuqPyCJgnczh5RcVeS5mi5UJydLzeVvcGcccd4 
IPFS Alternate Link: http://ipfs.biblepay.org:8080/ipfs/QmdMBtPQvuqPyCJgnczh5RcVeS5mi5UJydLzeVvcGcccd4 

Transaction ID: a9f2bbf8855aa9aa71e6de5cc7397311fc82f880900a2c49ec74bd1793d9366d-000

Debug information

Debit: -5316.68724846 tBiblepay
Debit: -5316.68700926 tBiblepay
Credit: 633.37325772 tBiblepay

Transaction:
CTransaction(hash=a9f2bbf885, ver=1, vin.size=2, vout.size=2, nLockTime=54656)
    CTxIn(COutPoint(aaa458f8c797caff2c1496523284a76a3b7221abbfdca63aa6359c699dd3b025, 0), scriptSig=47304402200da7c97435fa16, nSequence=4294967294)
    CTxIn(COutPoint(9e369d963e90c5391f85c57a36422b4e910a084e473f12d6d3722b41c99ee95a, 0), scriptSig=4730440220774651f9309807, nSequence=4294967294)
    CTxOut(nValue=633.37325772, scriptPubKey=76a914b578b1aceae2ee88995f786f)
    CTxOut(nValue=10000.00000000, scriptPubKey=76a9149cff090c148949e19f4ed915)

Inputs:
yYk8sAG6VrArnyuy5fi6rBjQfEaamd69kq Amount=5316.68724846 tBiblepay IsMine=true IsWatchOnly=false
testnet ycQa1b7LZUUbNunvgtWhvxPyT5YhEcJr5D Amount=5316.68700926 tBiblepay IsMine=true IsWatchOnly=false

XML:
{change}1{/change}{PACK}{MT}ATTACHMENT{/MT}{MK}OUT_TX{/MK}{MV}QmdMBtPQvuqPyCJgnczh5RcVeS5mi5UJydLzeVvcGcccd4{/MV}{ipfshash}QmdMBtPQvuqPyCJgnczh5RcVeS5mi5UJydLzeVvcGcccd4{/ipfshash}{/PACK}{PACK}{MT}MESSAGE{/MT}{MK}OUT_TX{/MK}{MV} {/MV}{/PACK}
« Last Edit: September 03, 2018, 02:05:18 AM by sunk818 »
BH6oxjLkyz3z8FYpvU3ZR7PTZ31Xt9DkXZ


  • jaapgvk
  • Hero Member

    • 558


    • 31
    • September 01, 2017, 08:02:57 PM
    • Netherlands
    more
Re: TestNet Testing Thread - Test IPFS Integration (Decentralized IP File System)
« Reply #58 on: September 03, 2018, 02:09:27 AM »
Trying to figure out how to set up masternode for the first time. Getting close I hope.


I found this guide to be good for testnet:
https://forum.biblepay.org/index.php?topic=108.msg1444#msg1444

I just run the testnet-sactuary hot on my home linux rig.


  • Rob Andrews
  • Administrator

    • 4090


    • 97
    • June 05, 2017, 08:09:04 PM
    • Patmos, Island Of
    more
Re: TestNet Testing Thread - Test IPFS Integration (Decentralized IP File System)
« Reply #59 on: September 03, 2018, 07:21:34 AM »
Thanks! I looked into it and the path is correct now. I adjusted the config and tried to start the daemon. Got this error message:

Code: [Select]
Error: serveHTTPGateway: manet.Listen(/ip4/MYIP/tcp/8080) failed: listen tcp4 MYIP:8080: bind: cannot assign requested address
Also, I have joined team Gridcoin (teamid=12575) and have connected my wallet to Rosetta@home :)
Jaap, sounds like you are close.
This is a guess, but could you double check the IP and format you typed into the config?  I would first go to your host, and check your public IP address, and make sure it matches.  Also you can type 'ipfs daemon' (after stopping the daemon) and check to see the lines it prints out that it is binding your IP address to other ports; write down the IP that is the public one- see that it matches your host (is your host vultr?) if it does you may just have an error on your gateway entry; it may be case sensitive also, compare the original entry from another node before changing it to the new entry?

One other thing to see: maybe you are already running a service on 8080.  Try to stop the ipfs daemon first, then type 'telnet localhost 8080', see if it replies.  If it does you are running something else on the box, maybe another type of web server.  You would have to stop that other thing first, then start the 'ipfs daemon'.


EDIT:  I just saw your post above that you run the linux rig hot at home; thats probably the issue; if running from the house, you would need to open up the firewall port 8080 and forward it to the linux rig, then it should bind the port.  One side issue with running at home:  check to see if your sanc says ENABLED in our list - the new PODS enforcer requires a sanc to be ENABLED to check its PODS quality.



« Last Edit: September 03, 2018, 07:33:12 AM by Rob A. »