01:58:54
preland:
@rucknium: I should be ready to present some more of my findings next Wednesday
15:04:27
rucknium:
MRL meeting in this room in two hours.
16:59:53
rucknium:
Meeting time! https://github.com/monero-project/meta/issues/1321
16:59:57
rucknium:
1. Greetings
17:00:16
vtnerd:
hi
17:00:21
rbrunner:
Hello
17:00:25
gingeropolous:
howdy
17:02:02
plowsof:
👋
17:03:17
rucknium:
2. Updates. What is everyone working on?
17:04:22
rucknium:
me: Fixed the database lock issue for the Monero net scanner. Revising the webapp (Not linking to it now since it has a big error message :D )
17:05:06
vtnerd:
me: re-worked the lwsf<->lws<->monerod protocols for fcmp++ tree building. still works! currently looking into a bug about 0-conf reporting and fcmp++
17:05:46
gingeropolous:
me: working on monerosim. claude code is wow. might have just one-shotted the whole mining shim thing. although i got the generate blocks one working great as well, doesn't require any monero mods. waiting on MRC resources freeing up to test 1000 node scale up. can currently get 85 agents on 32GB ram. nodes sync, txs generate and relay, included in blocks.
17:06:18
articmine:
Hi
17:07:14
jbabb:cypherstack.com:
knee-deep in reviewing the carrot impl
17:08:58
DataHoarder:
implemented carrot tx proofs (generate, verify) on my libraries, doing some benchmarks on RandomX V2
17:10:30
rucknium:
3. Spy nodes (https://github.com/monero-project/meta/issues/1124).
17:11:38
rucknium:
I had planned to have the spy nodes webapp revised by now, but I'm getting an unexpected launch error on the server at the moment.
17:12:44
rucknium:
The network scanner seems to be working well again after I switched to a different Rust crate for connected with SQLite databases. It allows specifying a longer connection timeout so the database doesn't appear fatally locked to the tokio threads.
17:13:37
rucknium:
For finalizing the new ban list, we need to have the original signers, at least, sign it. IIRC, SethForPrivacy's Docker image requires that the signatures are valid.
17:14:10
rucknium:
The signers would be @boog900:monero.social , @syntheticbird:monero.social , @jeffro256:monero.social , and myself.
17:15:10
rucknium:
I usually don't like to do announcements on Friday or Monday. So we could aim to announce tomorrow or next on Tuesday.
17:15:46
rucknium:
We would want to hit Reddit, monero.town, Twitter at least with the announcement.
17:16:13
rucknium:
And the GitHub issue
17:17:15
gingeropolous:
+1
17:19:10
rucknium:
Anything more on spy nodes?
17:20:34
rucknium:
4. FCMP alpha stressnet (https://monero.town/post/6763165).
17:22:11
rucknium:
@gingeropolous:monero.social suggested that tx spam volume be decreased so that he can have all the RAM available on the big-RAM Monero Research Computing machine. The one with 1TB of RAM.
17:22:20
rucknium:
I think that would be fine
17:22:44
rucknium:
So that @gingeropolous:monero.social can run the scaled-up monerosim
17:23:31
gingeropolous:
it should be like 5-8 hrs is all i need
17:25:09
articmine:
With the new scaling parameters we need to consider a significantly higher penalty free for stressnet instead of just relying on spam.
17:25:09
articmine:
Otherwise it will take years to test for many of the identified issues with scaling
17:26:31
rucknium:
Yes. I think initially we thought that the actual FCMP hard fork scaling parameters should be used for beta stressnet, but now I think differently.
17:27:03
articmine:
The simplest is to increase the penalty free zone ZM
17:27:34
rucknium:
By the way, I checked the current long term median block weight for this stressnet. It is 664 KB. The long term median before stressnet started was 176 KB.
17:28:10
articmine:
Then keep the other parameters 8x on MN and 1.2x on ML the same
17:28:13
rucknium:
AFAIK, get_info doesn't give you the long term median, which is what I thought before. You have to query a block header.
17:29:50
gingeropolous:
@articmine:monero.social: , im hopeful monerosim allows for faster testing. so far i can run 6 hours of monero network in about 20 minutes.... faster if the hardware is better
17:30:15
DataHoarder:
I show that long term median on each block on my explorer like https://stressnet.p2pool.observer/block/d9399b53688d0bb8a2db9d4fe7f5168a9935b0b067430b8dbeb444a9f42e6dc7
17:30:37
DataHoarder:
Indeed comes from block headers
17:31:11
articmine:
@gingeropolous: So a factor of 18. That will certainly help
17:31:58
gingeropolous:
gonna need moar ram
17:32:27
boog900:
@rucknium: I think that value would just be that blocks long term weight not the median long term weight
17:32:49
boog900:
https://monero-book.cuprate.org/consensus_rules/blocks/weights.html#calculating-a-blocks-long-term-weight
17:34:19
rucknium:
@boog900:monero.social: What's the difference?
17:36:58
articmine:
The long term block weight can be significantly higher than the long term median, if growth lags behind the long term median.
17:36:58
articmine:
It can also be lower
17:39:08
rucknium:
Is there an RPC query that can get the median long term weight?
17:40:13
boog900:
A blocks long term weight is the value that is used in the long term blocks list to get the median long term weight
17:40:14
boog900:
In the same way a blocks weight is used in the short term list to get the short term median block weight
17:40:14
boog900:
You could get the long term median weight under some circumstances if the block is outside a certain range by multiplying or dividing by 1.7. For example before stressnet the median was almost certainly 176 * 1.7
17:41:54
rucknium:
@boog900:monero.social: Your answer is "no, not an easy way to get this info"
17:42:26
rucknium:
unless Mercury is in retrograde
17:44:02
articmine:
If the long term block weight is under the penalty free zone, then the long term median is the penalty free zone.
17:44:24
articmine:
On it has been given enough time to reset
17:46:44
boog900:
@rucknium: Well the blocks long term weight is bounded to the long term median divided by 1.7 and multiplied by 1.7 so you can get the value if you have a very small or very large block.
17:46:55
boog900:
But yeah I'm not sure if there is an API that just tells you the value
17:48:13
articmine:
With the new proposed changes this bound is 1.2
17:50:23
rucknium:
In the 26 November meeting @jberman:monero.social said
17:50:23
rucknium:
> personally I'd like to complete the alpha stressnet (reach a point where alpha stressnet is running smooth under reasonable conditions), ideally within the next 4 weeks. and then reopen a conversation on target dates
17:51:35
rucknium:
@jberman:monero.social and/or @jeffro256:monero.social , how close do you think is the end of alpha and beginning of beta stressnet?
17:51:35
jberman:
apologies, I messed up timing on this meeting
17:51:59
jeffro256:
Me too, hi
17:52:17
jbabb:cypherstack.com:
A "scalenet" with modifered parameters to make spamming much easier to spam in order to get better data on the scaling question makes sense to me, but as a subsequent testnet following this stressnet with its current params
17:52:35
jbabb:cypherstack.com:
s/spam/do
17:52:48
articmine:
@boog900: No the block weight can still be up to 100x ML under the current scaling and up to 16x ML under the proposed scaling
17:53:14
jeffro256:
https://github.com/seraphis-migration/monero/issues/166
17:53:25
boog900:
@articmine: The long term weight is a different value.
17:54:01
jberman:
Re: end of alpha: We have 1 more open PR for v1.5 that prevents OOM's during initial block download. It is an upstream issue, but the stressnet is triggering it for some. That PR is: https://github.com/seraphis-migration/monero/pull/275
17:54:05
articmine:
I've how many cycles of ML
17:54:27
boog900:
@boog900: https://github.com/monero-project/monero/blob/eac1b86bb2818ac552457380c9dd421fb8935e5b/src/cryptonote_core/blockchain.cpp#L4569
17:54:28
articmine:
Over how many cycles of ML is this long term weight calculated
17:55:30
articmine:
MzL can lag
17:55:43
jeffro256:
We've got the weight func modified in the staging branch, Berman is currently working on some changes to tx relay v2 with 0xfffc, we're not going to wait for carrot-derived wallets, hot/cold wallets is ready on PR #52, Unbiased hash-to-point is implemented and ready to merged into the staging branch, the other points are mostly resolved on the staging branch AFAIK
17:56:11
jberman:
I also PR'd changes to tx relay v2 that would be nice to have tested in alpha, since tx relay v2 is a fairly significant change affecting the daemon: https://github.com/0xFFFC0000/monero/pull/62
17:56:29
jeffro256:
So we basically just need to integrate the updated scaling parameters and tx relay v2 and it should be ready to go I think
17:57:06
jeffro256:
Yeah true, we need to upstream the span changes
17:57:32
jeffro256:
But we don't have to wait for them to be upstreamed to start the beta, yeah?
17:58:53
jberman:
Sorry I wasn't clear. We need that PR specfically for v1.5 of the alpha stressnet. I was just highlighting how that is an upstream issue (and a separate upstream PR is warranted as well)
18:00:34
jberman:
I was thinking we verify that v1.5 solves all the major outstanding alpha stressnet issues people have reported during testing (like OOM's) before moving to beta
18:01:24
jberman:
A few stressnet users have also reported new crashes even with a pre-release of v1.5, and I'm still working my way through those, starting with this segfault: https://github.com/seraphis-migration/monero/issues/258
18:01:48
jberman:
I'm almost done with a fix for that issue, and then it's on to this issue: https://github.com/seraphis-migration/monero/issues/277
18:01:54
jeffro256:
What path should we take if people keep experiencing some of these same errors? Is it not still worth moving to the beta stressnet and continue debugging there?
18:03:27
jberman:
It's hard to prioritize changes like scaling algo for beta when there are triggerable outstanding segfaults
18:05:33
articmine:
@jberman: I would suggest keeping the existing scaling algo until these issues are fixed
18:05:44
jeffro256:
I absolutely wouldn't push for production with outstanding segfault issues, but ostensibly the errors, if not fixed, will also occur on beta for similar reasons.
18:06:12
jeffro256:
If they occur for similar reasons, then debugging on these issues isn't impacted by moving to beta.
18:06:20
jberman:
I also think we need a solution this PR is addressing first and foremost before we should move on to beta, because OOM's have plagued alpha and this should take care of the last major known cause: https://github.com/seraphis-migration/monero/pull/275
18:06:32
articmine:
@jeffro256: If we change the scaling algo not right away
18:06:50
jeffro256:
If anything, the scaling issues triggering segfaults more often would only make it easier to debug.
18:07:42
jberman:
@jeffro256: these aren't issues with the FCMP++ integration i.e. they are issues with current production code is also why I think they deserve priority
18:09:07
jeffro256:
@jberman: Also, sorry I meant to review this yesterday, will do today.
18:10:07
jberman:
np thank you 🙏
18:10:58
jeffro256:
@jberman: That's fair, I understand why youbut also I think that we can do this largely in parallel since as you're saying FCMP++ scaling and these OOM errors
18:11:10
jeffro256:
oops
18:11:50
rucknium:
Anything more on stressnet?
18:12:36
jeffro256:
I understand why prioritizing this issue over FCMP++ scaling, but I just think that they're mostly orthogonal . But optimistically, if #275 closes up the OOM issues, then we can go full speed ahead :)
18:12:50
jeffro256:
Let's hope that happens
18:15:52
jberman:
I agree they're definitely orthogonal. It's just a matter of manpower really. I don't want to divert attention away from dealing with reliability issues like the OOM's / segfault. if we had someone else (like perfect daemon :) ) focusing on them, I would be content moving on to beta tasks like scaling
18:16:47
rucknium:
Has anyone heard anything from perfect_daemon? I assume, given his M.O., that he's working on a huge PR in secret.
18:16:53
jeffro256:
Nope
18:17:14
jeffro256:
I've tried to DM unsuccessfully
18:17:21
jberman:
I assume the same
18:17:28
jbabb:cypherstack.com:
FCMP testing itself is definitedly a priority over scaling questions. a separate "scalenet" that makes spamming much easier is a orthogonal/tangential to the goal of "just" getting FCMPs working, but it allows an avenue for those that're concerned about scaling issues to prove their point without so much effort invested into spambots.
18:17:30
vtnerd:
@ruckinum I was about to say if perfect-daemon can’t look at this, I probably could given that my fcmp++ changes are going well
18:18:16
vtnerd:
the question is whether we should re-write the span code. probably needs it, otoh its semi-working in production so
18:18:36
articmine:
@jbabb:cypherstack.com: This is where a significantly higher penalty free zone can be used
18:19:05
boog900:
I think the syncing code is up there with the worst parts of the daemon.
18:19:40
jberman:
fwiw, I was primarly referring to issues like https://github.com/seraphis-migration/monero/issues/258 and https://github.com/seraphis-migration/monero/issues/277 , but span code also ya
18:19:46
jbabb:cypherstack.com:
I agree overall that they're separate questions. And maybe it is too much effort just to investigate scaling, but if we want to lower the parameters that we've apparently achieved consensus on, a "scalenet" seems like a way to do that more easily than with the current spam approach
18:20:00
jberman:
I'm pretty close to done on 258 though
18:20:57
rucknium:
@jbabb:cypherstack.com: What do you mean? Would there be no spam on scalenet?
18:22:24
jbabb:cypherstack.com:
@rucknium: I mean that the current spam efforts are doing well to increase blocksize, but if we really want to break things, opening up the scaling params would make that a lot easier to do. I agree that it's basically tangential to FCMP work, tho
18:23:17
articmine:
@jbabb:cypherstack.com: I agree.
18:23:34
jbabb:cypherstack.com:
since we already have loose consensus on params that can be paired with the FCMP HF, if we don't want to lower them anymore, a "scalenet" may be unnecessary.
18:23:39
jberman:
one benefit to the current approach is that it's showing cracks in the system as they appear (and would appear in production conditions), and we tackle them in that priority order
18:24:12
rucknium:
I think we just need to decide what's the target block size to hit in the next stressnet. Then make it easy to hit it.
18:24:15
jberman:
e.g. issues 4mb blocks showed up fairly early, got tackled fairly early because of that
18:24:20
jbabb:cypherstack.com:
it may be unnecessary or something that's safe to defer until after the FCMP HF / beta FCMP work
18:25:10
articmine:
@jbabb:cypherstack.com: The advantage of a scalenet is to accelerate the process.
18:26:08
articmine:
Then we can see in say a few months an issue that may take several years
18:29:36
rucknium:
Anything else on stressnet?
18:33:49
jberman:
My take on status: release v1.5 (hopefully within the next days), see what if any major reliability issues remain, either work on those reliability issues or continue toward beta. Tx relay v2 at this point may be fine to kick to beta
18:35:46
rucknium:
We can end the meeting here. Thanks everyone.
18:35:54
articmine:
Thanks
18:41:18
DataHoarder:
As a note to the long term weights I should be able to display these on the explorer for the current tip, given I'm getting the miner data and calculating the expected growth inline
18:42:53
DataHoarder:
Scalenet sounds great plus a few tricks of ours, if the point is to specifically stress that part. Might be good to expect resetting it regularly (so we don't end with TiB of state) and can test new changes iteratively
19:06:43
rucknium:
The new version of https://moneronet.info is working now 😁
19:36:48
monerobull:matrix.org:
> 1 TB ram > <@rucknium> @gingeropolous:monero.social suggested that tx spam volume be decreased so that he can have all the RAM available on the big-RAM Monero Research Computing machine. The one with 1TB of RAM.
19:36:48
monerobull:matrix.org:
> And I always thought we are a poor project!
20:20:27
rucknium:
It was funded by a CCS proposal: https://ccs.getmonero.org/proposals/gingeropolous_1TB_MRC.html
20:20:27
rucknium:
Thank you donors!
21:04:44
321bob321:
Ram only node inbound