Bible Pay

Read 216065 times

  • oncoapop
  • Full Member

    • 171


    • 17
    • October 23, 2018, 12:31:17 PM
    more
Re: TestNet - BiblePay-Evolution & GSCs (Generic Smart Contracts)
« Reply #120 on: April 01, 2019, 08:19:07 PM »
1.4.0.8-TestNet Mandatory Upgrade

- Add Watchman On the Wall (BiblePay version)
- Modify Sanctuary IsSuperblockValid trigger rule to trigger when
fCachedFunding is set
- Add rpc command 'getpogpoints' (this shows the pog points for a
transaction)

Sorry but I am unable to start watchman. can you please advise?

-342: non-JSON HTTP response with '401 Unauthorized' from server
Cannot connect to biblepayd. Please ensure biblepayd is running and the JSONRPC port is open to watchman.



  • Rob Andrews
  • Administrator

    • 4097


    • 97
    • June 05, 2017, 08:09:04 PM
    • Patmos, Island Of
    more
Re: TestNet - BiblePay-Evolution & GSCs (Generic Smart Contracts)
« Reply #121 on: April 02, 2019, 08:50:11 AM »
Sorry but I am unable to start watchman. can you please advise?

-342: non-JSON HTTP response with '401 Unauthorized' from server
Cannot connect to biblepayd. Please ensure biblepayd is running and the JSONRPC port is open to watchman.
Sorry for providing too little information about the Watchman environment; let me try to explain.

In Prod, we are running a 1.5 year old codebase with Dashs Sentinel in Python (we renamed it Watchman-on-the-wall) - and we still use it in prod as-is from Dash (and its still required today since it helps create the budgets etc, creates watchdogs, etc).

In Evo, I have not ported Sentinel to Watchman yet (so its unavailable still), so we do not have to Test it yet.

What I'm considering however, is porting Watchman to c++, so we don't have to run both watchman and Evo as separate programs in each sanc (in the future).  (Thats what this release info was about, but - its really just in step 1).  I'm still going to do some internal testing on my end, then provide more info as how we can test the integrated version of watchman, so please don't worry about testing that yet.






  • Rob Andrews
  • Administrator

    • 4097


    • 97
    • June 05, 2017, 08:09:04 PM
    • Patmos, Island Of
    more
Re: TestNet - BiblePay-Evolution & GSCs (Generic Smart Contracts)
« Reply #122 on: April 02, 2019, 09:30:08 AM »
** Update **

So for the last few re-releases the common theme has been:
- No signs of a problem with abn weight, or gsc client side transactions, or creation of the pog points
- No signs of problems with actual POBH heat mining (IE no forks resulting from POW)
- No signs of problems on the *monthly* superblock budget system or its creation of budget data

- Every re-release so far and enhancement resulted from a problem related to the Daily superblock contract
 (not a problem with the actual contract, a problem with the sancs not *finding* the contract, some find it, some lose it)

So I've been trying to hone in on this for the last 2 days - because at first I believed running only 3 sancs meant that one sanc out of 3 was not syncing properly- so we added rules to strengthen the minimum requirements of the voting required for a superblock to pass.  Yesterday however, all 3 sancs could see the votes on the superblock (we had 4), but only 2 of them had the actual gobject in memory (that is obviously a problem).

Let me explain this a different way, so we can all understand how a soft consensus can still fork the chain in testnet.
Sanc #1 and Sanc #2 had the gobject in memory (this is the GSC contract with its payments and addresses).  Sanc 1,2,and 3 had the *vote* for the gobject in memory and they all agreed the superblock was good and should pass.

Here is what happened on block 6990:

Sanc #1 found both the gobject and the trigger votes, and it said:  Valid, accepted
Sanc #2 found the votes only, and said "valid superblock at this height" but actually rejected the block because:  IsBlockValueValid, too few superblock payments <-- this is because it was missing the gobject with the hex data

So lets see what happens; Sanc #2 goes on its own chain now, Sanc 1 and 3 continue on the chain with the higher work.  This actually splits the testnet network because now Sanctuary votes for 1 cause (mnpayments to decrease), IE do to our mnpayments relying on chainheights- this becomes a hard split - and in a small
 testnet network we cant have this, so therefore we have to fix the root of the problem.

This ends up causing a hard fork, because #2 will never re-correct its view of this block until completely resynced as it has now marked the block as a fail.

First, lets stop testing again until I can release a plan to fix this.
(It does not appear to be watchman).
It appears to be more of a timing issue.  In Dash, the gobjects are synced over a longer time span (as we normally have a month to get all the data for the monthly votes), but in our case, we are trying to sync (with 2 minute blocks) all the superblock data within 190 blocks (IE < 3 hours)-- so I can see if one sanc misses a gobject, its going to make a bad call on the next superblock.

So, my plan is this next:
1. I will investigate all code that syncs gobjects, and see if we are lacking something (Ie timing issues) and beef this up
2. I will do something special to allow us to Monitor the next daily gsc superblock, like a command line that will alert us that a node is going to make a bad call on the next block
3. We will make the sanc go out of its way to Pull the data it needs to make a good call on the block manually

Then we regroup here and do an in-depth analysis on the actual superblock heights as they are about to tick by and post something on the forum to see what the state of each one is as they occur. 

Please hang on for the next version, it might take a day, it might take less depending on how complex the timing issue is.






  • Rob Andrews
  • Administrator

    • 4097


    • 97
    • June 05, 2017, 08:09:04 PM
    • Patmos, Island Of
    more
Re: TestNet - BiblePay-Evolution & GSCs (Generic Smart Contracts)
« Reply #123 on: April 02, 2019, 01:39:01 PM »
1.4.0.9-TestNet Mandatory Upgrade

- Remove more log spam
- Add 'exec health'; we use this to check the health of the next GSC
superblock manually
- Make the Sanc quorum relay the crown jewel once every 9 blocks
- Make the Sanc or the non-sanc detect an unhealthy GSC condition, and
Reset resyncing gov objs
- Prevent d-dossing other sancs for having a bad previous block header
- Lengthen gobject deletion time by 1 day


  • Rob Andrews
  • Administrator

    • 4097


    • 97
    • June 05, 2017, 08:09:04 PM
    • Patmos, Island Of
    more
Re: TestNet - BiblePay-Evolution & GSCs (Generic Smart Contracts)
« Reply #124 on: April 02, 2019, 03:15:05 PM »

16:14:22

exec health


16:14:22

{
  "Command": "health",
  "govobjhash": "0000000000000000000000000000000000000000000000000000000000000000",
  "Amounts": "",
  "Addresses": "",
  "votes": 0,
  "required_votes": 3,
  "last_superblock": 8835,
  "next_superblock": 9040,
  "next_superblock_triggered": false,
  "Healthy": false,
  "GSC_Voted_In": false
}


16:14:24

getblockhash 9000


16:14:24

eca1364b2b58b06d07aaad98685382b1e67b3a053a3db26c77bfddd61042fdd8


If all goes well, we will all go to "healthy".



  • Rob Andrews
  • Administrator

    • 4097


    • 97
    • June 05, 2017, 08:09:04 PM
    • Patmos, Island Of
    more
Re: TestNet - BiblePay-Evolution & GSCs (Generic Smart Contracts)
« Reply #125 on: April 02, 2019, 04:08:27 PM »
1.4.0.9b-TestNet Mandatory Upgrade

- Remove more log spam
- Add 'exec health'; we use this to check the health of the next GSC
superblock manually
- Make the Sanc quorum relay the crown jewel once every 9 blocks
- Make the Sanc or the non-sanc detect an unhealthy GSC condition, and
Reset resyncing gov objs
- Prevent d-dossing other sancs for having a bad previous block header
- Lengthen gobject deletion time by 1 day


** All, please ensure you upgrade again to the newest version **


  • Rob Andrews
  • Administrator

    • 4097


    • 97
    • June 05, 2017, 08:09:04 PM
    • Patmos, Island Of
    more
Re: TestNet - BiblePay-Evolution & GSCs (Generic Smart Contracts)
« Reply #126 on: April 02, 2019, 04:41:38 PM »
Ok I went to healthy on all my nodes, lets hope you all see the same.

'exec health'



  • uptimeminer
  • Newbie

    • 5


    • 1
    • October 22, 2018, 12:17:41 AM
    more
Re: TestNet - BiblePay-Evolution & GSCs (Generic Smart Contracts)
« Reply #127 on: April 02, 2019, 10:32:52 PM »
Switched to healthy=true on my node..

Quote
"Command": "health",
  "govobjhash": "a2ce1dd9f8fcb55ed32d62b3a071338928dc85f812f207ae918e04c38d49b4e1",
  "Amounts": "594237.00|299260.00|359384.00",
  "Addresses": "yS9YaCFpQuE6xpBrTvUbnPyJgwaXZep8C8|yUNSEjjtC9pdeHp4spswdFWh1npfV5Jvqe|ygavD5YuAJXpHDMLRyiiJ199YM5y9fTWfB",
  "votes": 3,
  "required_votes": 3,
  "last_superblock": 9040,
  "next_superblock": 9245,
  "next_superblock_triggered": false,
  "Healthy": true,
  "GSC_Voted_In": true


  • Rob Andrews
  • Administrator

    • 4097


    • 97
    • June 05, 2017, 08:09:04 PM
    • Patmos, Island Of
    more
Re: TestNet - BiblePay-Evolution & GSCs (Generic Smart Contracts)
« Reply #128 on: April 03, 2019, 05:30:45 AM »
Switched to healthy=true on my node..
Yes, thank God we finally made it through the entire night and all the superblocks passed all the rules this time (we had about 4 so far I think).


getblockhash 10120

e13331a79a04776297f1ef66634525bb3d16aca8c9ff8d61a5be21f2c5202caf

I can see we are all still in sync also.  I forgot to mention, you can see by the difficulty that the chain is solid now as it jumped up to 1.4 (from .10 when we had the issues).


Great, now I can take a look at the proposal rules for the monthlies and see if we can expose the height, and test watchman. 


I'm going to be explaining a (what I consider to be) novel use of our new campaigns later today.

We can discuss bearing fruit in Jesus Name with Healing campaigns and spiritual warfare in Evo. 

I'm thinking we can add a CPK startup wizard, with the ability to read an instruction PDF, join a campaign that requires directions (as spiritual warfare for instance is dangerous if a person does not know what they are doing), and once they join, we also need the ability for a user to type in a days activity back into BiblePay.    This would allow the campaign to keep a diary of what the user is doing for the points (for these specific campaigns).





  • Rob Andrews
  • Administrator

    • 4097


    • 97
    • June 05, 2017, 08:09:04 PM
    • Patmos, Island Of
    more
Re: TestNet - BiblePay-Evolution & GSCs (Generic Smart Contracts)
« Reply #129 on: April 03, 2019, 08:37:33 AM »
I've got a fun project coming up for us. 

So basically, here are the requirements for Watchman On The Wall we need to fulfill for replacement of The Sentinel, and be the first coin with our own integrated Watchman (not requiring python):

- Nodes get banned and not paid if they are not running watchman (this prevents people from running sancs that do not help vote on the budget)
- Nodes do not get voted on for a sanc payment if they are down (IE watchdog expired)
- Nodes who do run watchman can create a budget, sort it by votes, exclude proposals that pay dont fit, and vote on the proposal automatically during the warning period

We've got the first two covered (Watchman will automatically run in our sancs in c++, sancs will only vote on sancs who are Up (as we do test for that already), and we already have a spork for payment enforcement), and by the end of our testing, we will ensure Sanc don't expire prematurely--  so in light of all this, all that is left (the elephant in the room), is to test Watchman Budgeting.

In light of that, I am now modifying Watchman to handle budgets.  Where you all will come in to help, is we need to create about 10 *new* proposals (as the old ones are bad) -- for the next superblock (see 'getgovernanceinfo' to see the next budget is at height 11480, with getsuperblockbudget 11480 = 4.1MM bbp).

What we need to do in the future is : Verify proposals that missed a prior superblock deadline are no longer in the proposals list (this is broken currently), that future unpaid proposals are in the proposals list, and that watchman picks up these for budgeting, and sorts them by votes descending, and also excludes items that dont fit the cap.

What I want to do is have us all vote on these, and ensure the ones with low votes are not included and those that are too high for the budget cap are excluded automatically.

The current version *is* capable of taking new proposal now - for block 11480, so please, start now by creating 5-7 new proposals (please only create One at a time and then ensure the block has passed before creading another one).  I think if each of you create 2-3 that is sufficient.

Ill create a few myself now.

After the next release, the bugs above will be fixed and then we can test a watchman dry run at a height of 10% before block 11480 (that is the warning period) (IE 115 blocks early).





  • oncoapop
  • Full Member

    • 171


    • 17
    • October 23, 2018, 12:31:17 PM
    more
Re: TestNet - BiblePay-Evolution & GSCs (Generic Smart Contracts)
« Reply #130 on: April 03, 2019, 01:17:01 PM »
** Update **

So for the last few re-releases the common theme has been:
- No signs of a problem with abn weight, or gsc client side transactions, or creation of the pog points
- No signs of problems with actual POBH heat mining (IE no forks resulting from POW)
- No signs of problems on the *monthly* superblock budget system or its creation of budget data

- Every re-release so far and enhancement resulted from a problem related to the Daily superblock contract
 (not a problem with the actual contract, a problem with the sancs not *finding* the contract, some find it, some lose it)

So I've been trying to hone in on this for the last 2 days - because at first I believed running only 3 sancs meant that one sanc out of 3 was not syncing properly- so we added rules to strengthen the minimum requirements of the voting required for a superblock to pass.  Yesterday however, all 3 sancs could see the votes on the superblock (we had 4), but only 2 of them had the actual gobject in memory (that is obviously a problem).

Let me explain this a different way, so we can all understand how a soft consensus can still fork the chain in testnet.
Sanc #1 and Sanc #2 had the gobject in memory (this is the GSC contract with its payments and addresses).  Sanc 1,2,and 3 had the *vote* for the gobject in memory and they all agreed the superblock was good and should pass.

Here is what happened on block 6990:

Sanc #1 found both the gobject and the trigger votes, and it said:  Valid, accepted
Sanc #2 found the votes only, and said "valid superblock at this height" but actually rejected the block because:  IsBlockValueValid, too few superblock payments <-- this is because it was missing the gobject with the hex data

So lets see what happens; Sanc #2 goes on its own chain now, Sanc 1 and 3 continue on the chain with the higher work.  This actually splits the testnet network because now Sanctuary votes for 1 cause (mnpayments to decrease), IE do to our mnpayments relying on chainheights- this becomes a hard split - and in a small
 testnet network we cant have this, so therefore we have to fix the root of the problem.

This ends up causing a hard fork, because #2 will never re-correct its view of this block until completely resynced as it has now marked the block as a fail.

First, lets stop testing again until I can release a plan to fix this.
(It does not appear to be watchman).
It appears to be more of a timing issue.  In Dash, the gobjects are synced over a longer time span (as we normally have a month to get all the data for the monthly votes), but in our case, we are trying to sync (with 2 minute blocks) all the superblock data within 190 blocks (IE < 3 hours)-- so I can see if one sanc misses a gobject, its going to make a bad call on the next superblock.

So, my plan is this next:
1. I will investigate all code that syncs gobjects, and see if we are lacking something (Ie timing issues) and beef this up
2. I will do something special to allow us to Monitor the next daily gsc superblock, like a command line that will alert us that a node is going to make a bad call on the next block
3. We will make the sanc go out of its way to Pull the data it needs to make a good call on the block manually

Then we regroup here and do an in-depth analysis on the actual superblock heights as they are about to tick by and post something on the forum to see what the state of each one is as they occur. 

Please hang on for the next version, it might take a day, it might take less depending on how complex the timing issue is.

I appear to be on the correct chain by as far as I can tell (checking manually), I have not seen my sanc (the 4th) vote for any superblock. Does this have to be done manually?

bbpd@sanc2:~/Sanctuary$ cli getblockhash 10120
e13331a79a04776297f1ef66634525bb3d16aca8c9ff8d61a5be21f2c5202caf
bbpd@sanc2:~/Sanctuary$ cli exec health
{
  "Command": "health",
  "govobjhash": "9ff38ea222d3bb56085b2c696a10ddf02d16548cdb23beec3db828ae5ed8b929",
  "Amounts": "736194.00|180067.00|173696.00|101790.00|23491.00|37641.00",
  "Addresses": "yR26BVwFeGTnkgdfw1RRNgVF2LyPnrZqaT|yS9YaCFpQuE6xpBrTvUbnPyJgwaXZep8C8|yUNSEjjtC9pdeHp4spswdFWh1npfV5Jvqe|yfqGyVvuyidYytq5o2QvN1VdVeXtH9Lrkt|ygavD5YuAJXpHDMLRyiiJ199YM5y9fTWfB|yiWrpMBA8YXfVmxntAgZZ1dh18ueaBerbW",
  "votes": 2,
  "required_votes": 3,
  "last_superblock": 10680,
  "next_superblock": 10885,
  "next_superblock_triggered": true,
  "Healthy": true,
  "GSC_Voted_In": false
}


  • Rob Andrews
  • Administrator

    • 4097


    • 97
    • June 05, 2017, 08:09:04 PM
    • Patmos, Island Of
    more
Re: TestNet - BiblePay-Evolution & GSCs (Generic Smart Contracts)
« Reply #131 on: April 03, 2019, 01:58:18 PM »
I appear to be on the correct chain by as far as I can tell (checking manually), I have not seen my sanc (the 4th) vote for any superblock. Does this have to be done manually?

bbpd@sanc2:~/Sanctuary$ cli getblockhash 10120
e13331a79a04776297f1ef66634525bb3d16aca8c9ff8d61a5be21f2c5202caf
bbpd@sanc2:~/Sanctuary$ cli exec health
{
  "Command": "health",
  "govobjhash": "9ff38ea222d3bb56085b2c696a10ddf02d16548cdb23beec3db828ae5ed8b929",
  "Amounts": "736194.00|180067.00|173696.00|101790.00|23491.00|37641.00",
  "Addresses": "yR26BVwFeGTnkgdfw1RRNgVF2LyPnrZqaT|yS9YaCFpQuE6xpBrTvUbnPyJgwaXZep8C8|yUNSEjjtC9pdeHp4spswdFWh1npfV5Jvqe|yfqGyVvuyidYytq5o2QvN1VdVeXtH9Lrkt|ygavD5YuAJXpHDMLRyiiJ199YM5y9fTWfB|yiWrpMBA8YXfVmxntAgZZ1dh18ueaBerbW",
  "votes": 2,
  "required_votes": 3,
  "last_superblock": 10680,
  "next_superblock": 10885,
  "next_superblock_triggered": true,
  "Healthy": true,
  "GSC_Voted_In": false
}

I think things are OK on the back end, but we might need another field - but first whats your sanc Ip, I can check some things to make sure I see your vote?

Btw, no - you should not have to vote manually, this is all automatic.

« Last Edit: April 03, 2019, 02:00:04 PM by Rob Andrews »


  • oncoapop
  • Full Member

    • 171


    • 17
    • October 23, 2018, 12:31:17 PM
    more
Re: TestNet - BiblePay-Evolution & GSCs (Generic Smart Contracts)
« Reply #132 on: April 03, 2019, 02:14:44 PM »
I think things are OK on the back end, but we might need another field - but first whats your sanc Ip, I can check some things to make sure I see your vote?

Btw, no - you should not have to vote manually, this is all automatic.

Thanks.

bbpd@sanc2:~$ ifconfig
ens160: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 45.62.239.200  netmask 255.255.255.0  broadcast 45.62.239.255
        inet6 fe80::250:56ff:fe95:4682  prefixlen 64  scopeid 0x20<link>
        ether 00:50:56:95:46:82  txqueuelen 1000  (Ethernet)
        RX packets 600708  bytes 190734759 (190.7 MB)
        RX errors 0  dropped 276  overruns 0  frame 0
        TX packets 530771  bytes 225023604 (225.0 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0


  • Rob Andrews
  • Administrator

    • 4097


    • 97
    • June 05, 2017, 08:09:04 PM
    • Patmos, Island Of
    more
Re: TestNet - BiblePay-Evolution & GSCs (Generic Smart Contracts)
« Reply #133 on: April 03, 2019, 02:35:48 PM »
Thanks.

bbpd@sanc2:~$ ifconfig
ens160: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 45.62.239.200  netmask 255.255.255.0  broadcast 45.62.239.255
        inet6 fe80::250:56ff:fe95:4682  prefixlen 64  scopeid 0x20<link>
        ether 00:50:56:95:46:82  txqueuelen 1000  (Ethernet)
        RX packets 600708  bytes 190734759 (190.7 MB)
        RX errors 0  dropped 276  overruns 0  frame 0
        TX packets 530771  bytes 225023604 (225.0 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0


Yes, I see you as the 4th enabled sanc, great.

So trying to track this down took me longer than when it actually passed, so I found that it would be useful for us to have a couple more features.
1- The ability to see that the exec testgscvote is actually picking the most popular contract; right now I see 2 in there and it appears to be voting on the right one, but we cant really see it from the rpc
2- A way to filter the gobject get with a wildcard. 

Let me enhance the next version then we can test another one.



  • Rob Andrews
  • Administrator

    • 4097


    • 97
    • June 05, 2017, 08:09:04 PM
    • Patmos, Island Of
    more
Re: TestNet - BiblePay-Evolution & GSCs (Generic Smart Contracts)
« Reply #134 on: April 03, 2019, 05:19:07 PM »
1.4.1.0-Leisure Upgrade for TestNet

- Add cascading GSC superblock creation (this decreases the propensity
for duplicate GSCs)
- Add intelligent GSC voting system (this means we vote yes on the
lowest PAM hashes and vote no on the highest PAM hashes)
- Add Watchman on the Wall budgeting
- Add 'gobject listwild all type wildcard' - This allows us to search
objects or proposals or gscs or triggers by wild card.  You can also
search by gov plain text data or by hash.
- Add getgschashes - This gives us a list of current gsc hashes for the
next superblock and pertinent data for each hash