Minecon724/dncurrency
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages 1 Wiki Activity Actions 1

Experimental ledger pruning (#2881)

Browse source 
This is the initial, experimental implementation of pruning, related to https://github.com/nanocurrency/nano-node/issues/1094. Node ledger pruning is designed to safely discard parts of the ledger. Now ledger size can be reduced from 23.8 GB to 7.5GB after vacuum. And from 18.8 GB to 5.2 GB after database rebuild.

Configuration and considerations
- This version of ledger pruning is for experimental use only:
- Enable optional ledger pruning with node CLI `--debug_prune` for manual pruning & node launch flag `--enable_pruning` for automatic node background actions.
- Pruning rules can be configured in [node.experimental] config with “max_pruning_age” (Time in seconds to limit for blocks age after pruning. Also period for automatic node pruning task, default 1 day, 5 minutes for beta network) & “max_pruning_depth” (Limit for full blocks in chain after pruning, default 0 - unlimited). By default all confirmed blocks older than 1 day can be pruned, except genesis block & last confirmed block in each account chain.
- Last confirmed block in account chain & all unconfirmed blocks cannot be pruned from ledger. Pruning is relying on confirmation height, so confirmation height reset or too low online_weight_minimum config can cause issues.
- Pruned blocks hashes are located in database table “pruned” & can be used by node internally (https://github.com/nanocurrency/nano-node/pull/2946).

RPC, CLI and other changes
- Several RPC calls are modified to support pruned blocks related to main request block (https://github.com/nanocurrency/nano-node/pull/2977).
- RPC “history”: if previous block is pruned, “amount” field is not returned for all blocks, “account” field is not returned for state blocks, also state block type/subtype is “unknown” . If source is pruned, then “account” field is not returned for receive/open legacy & for receive/open state blocks (https://github.com/nanocurrency/nano-node/pull/2977).
- Same as above for QT wallet history (https://github.com/nanocurrency/nano-node/pull/2977).
- RPC “block_info”/“blocks_info”: if previous block is pruned, “amount” field is not returned. RPC “block_count”: for nodes with enabled pruning returns 2 new fields: “full” blocks & “pruned” blocks. “full” + “pruned” = total “blocks” (https://github.com/nanocurrency/nano-node/pull/2977).
- Block count for CLI/RPC/telemetry is sum of full blocks & pruned blocks (https://github.com/nanocurrency/nano-node/pull/2977).
- RPC "pruned_exists" to chekc pruned block existence in local database.

Other notes
- Node is aggressively pruning ledger during initial bootstrap (until hardcoded block count) to limit node disk space usage, pruning config settings are ignored. If you want to avoid this behavior, finish full bootstrap before pruning start.
-  `--enable_pruning` flag & config option “enable_voting” are mutually exclusive to prevent representatives start with pruned ledger. Pruning is not available for any representatives (https://github.com/nanocurrency/nano-node/pull/2947).
- Pruned node can serve as bootstrap server for remaining blocks in ledger (https://github.com/nanocurrency/nano-node/pull/2976).
- Multiple internal functions are modified to use pruned blocks in block processor (https://github.com/nanocurrency/nano-node/pull/2974), internal node wallet actions (https://github.com/nanocurrency/nano-node/pull/2992), confirmation height processor (https://github.com/nanocurrency/nano-node/pull/2978) and rep crawler (https://github.com/nanocurrency/nano-node/pull/2975).
- Development: consider using ledger safe functions for code with possible pruned blocks (https://github.com/nanocurrency/nano-node/pull/2968).

Additional updates being reviewed
- Include pruned count in telemetry.
- Improve RocksDB pruned block count calculation.

Co-authored-by: Wesley Shillingford <650038+wezrule@users.noreply.github.com>
Co-authored-by: Guilherme Lawless <guilherme@nano.org>
This commit is contained in:
Sergey Kroshnin 2020-11-06 21:04:19 +03:00 • committed by GitHub
commit 8e5f4262c2
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
11 changed files with 396 additions and 12 deletions
Download patch file Download diff file Expand all files Collapse all files

15
nano/core_test/block.cpp
View file

@ -503,6 +503,9 @@ TEST (block_builder, state)
.link_hex ("E16DD58C1EFA8B521545B0A74375AA994D9FC43828A4266D75ECF57F07A7EE86")
.build (ec);
ASSERT_EQ (block->hash ().to_string (), "2D243F8F92CDD0AD94A1D456A6B15F3BE7A6FCBD98D4C5831D06D15C818CD81F");
ASSERT_TRUE (block->source ().is_zero ());
ASSERT_TRUE (block->destination ().is_zero ());
ASSERT_EQ (block->link ().to_string (), "E16DD58C1EFA8B521545B0A74375AA994D9FC43828A4266D75ECF57F07A7EE86");
}
TEST (block_builder, state_missing_rep)
@ -573,6 +576,9 @@ TEST (block_builder, open)
.source_hex ("E89208DD038FBB269987689621D52292AE9C35941A7484756ECCED92A65093BA")
.build (ec);
ASSERT_EQ (block->hash ().to_string (), "991CF190094C00F0B68E2E5F75F6BEE95A2E0BD93CEAA4A6734DB9F19B728948");
ASSERT_EQ (block->source ().to_string (), "E89208DD038FBB269987689621D52292AE9C35941A7484756ECCED92A65093BA");
ASSERT_TRUE (block->destination ().is_zero ());
ASSERT_TRUE (block->link ().is_zero ());
}
TEST (block_builder, open_equality)
@ -609,6 +615,9 @@ TEST (block_builder, change)
.previous_hex ("088EE46429CA936F76C4EAA20B97F6D33E5D872971433EE0C1311BCB98764456")
.build (ec);
ASSERT_EQ (block->hash ().to_string (), "13552AC3928E93B5C6C215F61879358E248D4A5246B8B3D1EEC5A566EDCEE077");
ASSERT_TRUE (block->source ().is_zero ());
ASSERT_TRUE (block->destination ().is_zero ());
ASSERT_TRUE (block->link ().is_zero ());
}
TEST (block_builder, change_equality)
@ -645,6 +654,9 @@ TEST (block_builder, send)
.balance_hex ("00F035A9C7D818E7C34148C524FFFFEE")
.build (ec);
ASSERT_EQ (block->hash ().to_string (), "4560E7B1F3735D082700CFC2852F5D1F378F7418FD24CEF1AD45AB69316F15CD");
ASSERT_TRUE (block->source ().is_zero ());
ASSERT_EQ (block->destination ().to_account (), "nano_1gys8r4crpxhp94n4uho5cshaho81na6454qni5gu9n53gksoyy1wcd4udyb");
ASSERT_TRUE (block->link ().is_zero ());
}
TEST (block_builder, send_equality)
@ -704,4 +716,7 @@ TEST (block_builder, receive)
.source_hex ("7B2B0A29C1B235FDF9B4DEF2984BB3573BD1A52D28246396FBB3E4C5FE662135")
.build (ec);
ASSERT_EQ (block->hash ().to_string (), "6C004BF911D9CF2ED75CF6EC45E795122AD5D093FF5A83EDFBA43EC4A3EDC722");
ASSERT_EQ (block->source ().to_string (), "7B2B0A29C1B235FDF9B4DEF2984BB3573BD1A52D28246396FBB3E4C5FE662135");
ASSERT_TRUE (block->destination ().is_zero ());
ASSERT_TRUE (block->link ().is_zero ());
}

15
nano/core_test/bootstrap.cpp
View file

@ -727,10 +727,7 @@ TEST (bootstrap_processor, lazy_hash_pruning)
}
ASSERT_TIMELY (2s, node1->active.empty () && node1->block_confirmed (change2->hash ()));
// Pruning action
{
auto transaction (node1->store.tx_begin_write ());
ASSERT_EQ (3, node1->ledger.pruning_action (transaction, change1->hash (), 1));
}
node1->ledger_pruning (2, false, false);
ASSERT_EQ (9, node0->ledger.cache.block_count);
ASSERT_EQ (0, node0->ledger.cache.pruned_count);
ASSERT_EQ (5, node1->ledger.cache.block_count);
@ -931,10 +928,7 @@ TEST (bootstrap_processor, lazy_pruning_missing_block)
}
ASSERT_TIMELY (2s, node1->active.empty () && node1->block_confirmed (state_open->hash ()));
// Pruning action
{
auto transaction (node1->store.tx_begin_write ());
ASSERT_EQ (1, node1->ledger.pruning_action (transaction, send1->hash (), 1));
}
node1->ledger_pruning (2, false, false);
ASSERT_EQ (5, node1->ledger.cache.block_count);
ASSERT_EQ (1, node1->ledger.cache.pruned_count);
ASSERT_FALSE (node1->ledger.block_exists (send1->hash ()));
@ -1362,10 +1356,7 @@ TEST (bulk, genesis_pruning)
election->force_confirm ();
}
ASSERT_TIMELY (2s, node1->active.empty () && node1->block_confirmed (send3->hash ()));
{
auto transaction (node1->store.tx_begin_write ());
ASSERT_EQ (2, node1->ledger.pruning_action (transaction, send2->hash (), 2));
}
node1->ledger_pruning (2, false, false);
ASSERT_EQ (2, node1->ledger.cache.pruned_count);
ASSERT_EQ (4, node1->ledger.cache.block_count);
ASSERT_FALSE (node1->ledger.block_exists (send1->hash ()));

168
nano/core_test/node.cpp
View file

@ -4494,6 +4494,174 @@ TEST (rep_crawler, local)
}
}
TEST (node, pruning_automatic)
{
nano::system system;
nano::node_config node_config (nano::get_available_port (), system.logging);
node_config.max_pruning_age = std::chrono::seconds (1);
node_config.enable_voting = false; // Remove after allowing pruned voting
nano::node_flags node_flags;
node_flags.enable_pruning = true;
auto & node1 = *system.add_node (node_config, node_flags);
nano::genesis genesis;
nano::keypair key1;
auto send1 = nano::send_block_builder ()
.previous (genesis.hash ())
.destination (key1.pub)
.balance (nano::genesis_amount - nano::Gxrb_ratio)
.sign (nano::dev_genesis_key.prv, nano::dev_genesis_key.pub)
.work (*system.work.generate (genesis.hash ()))
.build_shared ();
auto send2 = nano::send_block_builder ()
.previous (send1->hash ())
.destination (key1.pub)
.balance (0)
.sign (nano::dev_genesis_key.prv, nano::dev_genesis_key.pub)
.work (*system.work.generate (send1->hash ()))
.build_shared ();
// Process as local blocks
node1.process_active (send1);
node1.process_active (send2);
node1.block_processor.flush ();
// Confirm last block to prune previous
{
auto election = node1.active.election (send1->qualified_root ());
ASSERT_NE (nullptr, election);
election->force_confirm ();
}
ASSERT_TIMELY (2s, node1.block_confirmed (send1->hash ()) && node1.active.active (send2->qualified_root ()));
ASSERT_EQ (0, node1.ledger.cache.pruned_count);
{
auto election = node1.active.election (send2->qualified_root ());
ASSERT_NE (nullptr, election);
election->force_confirm ();
}
ASSERT_TIMELY (2s, node1.active.empty () && node1.block_confirmed (send2->hash ()));
// Check pruning result
ASSERT_TIMELY (3s, node1.ledger.cache.pruned_count == 1);
ASSERT_TIMELY (2s, node1.store.pruned_count (node1.store.tx_begin_read ()) == 1); // Transaction commit
ASSERT_EQ (1, node1.ledger.cache.pruned_count);
ASSERT_EQ (3, node1.ledger.cache.block_count);
ASSERT_TRUE (node1.ledger.block_exists (genesis.hash ()));
ASSERT_FALSE (node1.ledger.block_exists (send1->hash ()));
ASSERT_TRUE (node1.ledger.block_or_pruned_exists (send1->hash ()));
ASSERT_TRUE (node1.ledger.block_exists (send2->hash ()));
}
TEST (node, pruning_age)
{
nano::system system;
nano::node_config node_config (nano::get_available_port (), system.logging);
node_config.enable_voting = false; // Remove after allowing pruned voting
nano::node_flags node_flags;
node_flags.enable_pruning = true;
auto & node1 = *system.add_node (node_config, node_flags);
nano::genesis genesis;
nano::keypair key1;
auto send1 = nano::send_block_builder ()
.previous (genesis.hash ())
.destination (key1.pub)
.balance (nano::genesis_amount - nano::Gxrb_ratio)
.sign (nano::dev_genesis_key.prv, nano::dev_genesis_key.pub)
.work (*system.work.generate (genesis.hash ()))
.build_shared ();
auto send2 = nano::send_block_builder ()
.previous (send1->hash ())
.destination (key1.pub)
.balance (0)
.sign (nano::dev_genesis_key.prv, nano::dev_genesis_key.pub)
.work (*system.work.generate (send1->hash ()))
.build_shared ();
// Process as local blocks
node1.process_active (send1);
node1.process_active (send2);
node1.block_processor.flush ();
// Confirm last block to prune previous
{
auto election = node1.active.election (send1->qualified_root ());
ASSERT_NE (nullptr, election);
election->force_confirm ();
}
ASSERT_TIMELY (2s, node1.block_confirmed (send1->hash ()) && node1.active.active (send2->qualified_root ()));
ASSERT_EQ (0, node1.ledger.cache.pruned_count);
{
auto election = node1.active.election (send2->qualified_root ());
ASSERT_NE (nullptr, election);
election->force_confirm ();
}
ASSERT_TIMELY (2s, node1.active.empty () && node1.block_confirmed (send2->hash ()));
// Pruning with default age 1 day
node1.ledger_pruning (1, true, false);
ASSERT_EQ (0, node1.ledger.cache.pruned_count);
ASSERT_EQ (3, node1.ledger.cache.block_count);
// Pruning with max age 0
node1.config.max_pruning_age = std::chrono::seconds (0);
node1.ledger_pruning (1, true, false);
ASSERT_EQ (1, node1.ledger.cache.pruned_count);
ASSERT_EQ (3, node1.ledger.cache.block_count);
ASSERT_TRUE (node1.ledger.block_exists (genesis.hash ()));
ASSERT_FALSE (node1.ledger.block_exists (send1->hash ()));
ASSERT_TRUE (node1.ledger.block_or_pruned_exists (send1->hash ()));
ASSERT_TRUE (node1.ledger.block_exists (send2->hash ()));
}
TEST (node, pruning_depth)
{
nano::system system;
nano::node_config node_config (nano::get_available_port (), system.logging);
node_config.enable_voting = false; // Remove after allowing pruned voting
nano::node_flags node_flags;
node_flags.enable_pruning = true;
auto & node1 = *system.add_node (node_config, node_flags);
nano::genesis genesis;
nano::keypair key1;
auto send1 = nano::send_block_builder ()
.previous (genesis.hash ())
.destination (key1.pub)
.balance (nano::genesis_amount - nano::Gxrb_ratio)
.sign (nano::dev_genesis_key.prv, nano::dev_genesis_key.pub)
.work (*system.work.generate (genesis.hash ()))
.build_shared ();
auto send2 = nano::send_block_builder ()
.previous (send1->hash ())
.destination (key1.pub)
.balance (0)
.sign (nano::dev_genesis_key.prv, nano::dev_genesis_key.pub)
.work (*system.work.generate (send1->hash ()))
.build_shared ();
// Process as local blocks
node1.process_active (send1);
node1.process_active (send2);
node1.block_processor.flush ();
// Confirm last block to prune previous
{
auto election = node1.active.election (send1->qualified_root ());
ASSERT_NE (nullptr, election);
election->force_confirm ();
}
ASSERT_TIMELY (2s, node1.block_confirmed (send1->hash ()) && node1.active.active (send2->qualified_root ()));
ASSERT_EQ (0, node1.ledger.cache.pruned_count);
{
auto election = node1.active.election (send2->qualified_root ());
ASSERT_NE (nullptr, election);
election->force_confirm ();
}
ASSERT_TIMELY (2s, node1.active.empty () && node1.block_confirmed (send2->hash ()));
// Pruning with default depth (unlimited)
node1.ledger_pruning (1, true, false);
ASSERT_EQ (0, node1.ledger.cache.pruned_count);
ASSERT_EQ (3, node1.ledger.cache.block_count);
// Pruning with max depth 1
node1.config.max_pruning_depth = 1;
node1.ledger_pruning (1, true, false);
ASSERT_EQ (1, node1.ledger.cache.pruned_count);
ASSERT_EQ (3, node1.ledger.cache.block_count);
ASSERT_TRUE (node1.ledger.block_exists (genesis.hash ()));
ASSERT_FALSE (node1.ledger.block_exists (send1->hash ()));
ASSERT_TRUE (node1.ledger.block_or_pruned_exists (send1->hash ()));
ASSERT_TRUE (node1.ledger.block_exists (send2->hash ()));
}
namespace
{
void add_required_children_node_config_tree (nano::jsonconfig & tree)

2
nano/lib/errors.cpp
View file

@ -211,6 +211,8 @@ std::string nano::error_rpc_messages::message (int ev) const
return "Invalid threads count";
case nano::error_rpc::peer_not_found:
return "Peer not found";
case nano::error_rpc::pruning_disabled:
return "Pruning is disabled";
case nano::error_rpc::requires_port_and_address:
return "Both port and address required";
case nano::error_rpc::rpc_control_disabled:

1
nano/lib/errors.hpp
View file

@ -118,6 +118,7 @@ enum class error_rpc
invalid_timestamp,
invalid_threads_count,
peer_not_found,
pruning_disabled,
requires_port_and_address,
rpc_control_disabled,
sign_hash_disabled,

10
nano/nano_node/entry.cpp
View file

@ -102,6 +102,7 @@ int main (int argc, char * const * argv)
("debug_account_versions", "Display the total counts of each version for all accounts (including unpocketed)")
("debug_unconfirmed_frontiers", "Displays the account, height (sorted), frontier and cemented frontier for all accounts which are not fully confirmed")
("validate_blocks,debug_validate_blocks", "Check all blocks for correct hash, signature, work value")
("debug_prune", "Prune accounts up to last confirmed blocks (EXPERIMENTAL)")
("platform", boost::program_options::value<std::string> (), "Defines the <platform> for OpenCL commands")
("device", boost::program_options::value<std::string> (), "Defines <device> for OpenCL command")
("threads", boost::program_options::value<std::string> (), "Defines <threads> count for various commands")
@ -1895,6 +1896,15 @@ int main (int argc, char * const * argv)
nano::inactive_node node (data_path, node_flags);
std::cout << "Total cemented block count: " << node.node->ledger.cache.cemented_count << std::endl;
}
else if (vm.count ("debug_prune"))
{
auto node_flags = nano::inactive_node_flag_defaults ();
node_flags.read_only = false;
nano::update_flags (node_flags, vm);
nano::inactive_node inactive_node (data_path, node_flags);
auto node = inactive_node.node;
node->ledger_pruning (node_flags.block_processor_batch_size != 0 ? node_flags.block_processor_batch_size : 16 * 1024, true, true);
}
else if (vm.count ("debug_stacktrace"))
{
std::cout << boost::stacktrace::stacktrace ();

20
nano/node/json_handler.cpp
View file

@ -3110,6 +3110,25 @@ void nano::json_handler::process ()
}));
}
void nano::json_handler::pruned_exists ()
{
auto hash (hash_impl ());
if (!ec)
{
auto transaction (node.store.tx_begin_read ());
if (node.ledger.pruning)
{
auto exists (node.store.pruned_exists (transaction, hash));
response_l.put ("exists", exists ? "1" : "0");
}
else
{
ec = nano::error_rpc::pruning_disabled;
}
}
response_errors ();
}
void nano::json_handler::receive ()
{
auto wallet (wallet_impl ());
@ -5041,6 +5060,7 @@ ipc_json_handler_no_arg_func_map create_ipc_json_handler_no_arg_func_map ()
no_arg_funcs.emplace ("pending", &nano::json_handler::pending);
no_arg_funcs.emplace ("pending_exists", &nano::json_handler::pending_exists);
no_arg_funcs.emplace ("process", &nano::json_handler::process);
no_arg_funcs.emplace ("pruned_exists", &nano::json_handler::pruned_exists);
no_arg_funcs.emplace ("receive", &nano::json_handler::receive);
no_arg_funcs.emplace ("receive_minimum", &nano::json_handler::receive_minimum);
no_arg_funcs.emplace ("receive_minimum_set", &nano::json_handler::receive_minimum_set);

1
nano/node/json_handler.hpp
View file

@ -84,6 +84,7 @@ public:
void pending ();
void pending_exists ();
void process ();
void pruned_exists ();
void receive ();
void receive_minimum ();
void receive_minimum_set ();

125
nano/node/node.cpp
View file

@ -650,6 +650,13 @@ void nano::node::start ()
this_l->ongoing_unchecked_cleanup ();
});
}
if (flags.enable_pruning)
{
auto this_l (shared ());
worker.push_task ([this_l]() {
this_l->ongoing_ledger_pruning ();
});
}
if (!flags.disable_rep_crawler)
{
rep_crawler.start ();
@ -974,6 +981,124 @@ void nano::node::ongoing_unchecked_cleanup ()
});
}
bool nano::node::collect_ledger_pruning_targets (std::deque<nano::block_hash> & pruning_targets_a, nano::account & last_account_a, uint64_t const batch_read_size_a, uint64_t const max_depth_a, uint64_t const cutoff_time_a)
{
uint64_t read_operations (0);
bool finish_transaction (false);
auto transaction (store.tx_begin_read ());
for (auto i (store.confirmation_height_begin (transaction, last_account_a)), n (store.confirmation_height_end ()); i != n && !finish_transaction;)
{
++read_operations;
auto const & account (i->first);
nano::block_hash hash (i->second.frontier);
uint64_t depth (0);
while (!hash.is_zero () && depth < max_depth_a)
{
auto block (store.block_get (transaction, hash));
if (block != nullptr)
{
if (block->sideband ().timestamp > cutoff_time_a || depth == 0)
{
hash = block->previous ();
}
else
{
break;
}
}
else
{
release_assert (depth != 0);
hash = 0;
}
if (++depth % batch_read_size_a == 0)
{
transaction.refresh ();
}
}
if (!hash.is_zero ())
{
pruning_targets_a.push_back (hash);
}
read_operations += depth;
if (read_operations >= batch_read_size_a)
{
last_account_a = account.number () + 1;
finish_transaction = true;
}
else
{
++i;
}
}
return !finish_transaction || last_account_a.is_zero ();
}
void nano::node::ledger_pruning (uint64_t const batch_size_a, bool bootstrap_weight_reached_a, bool log_to_cout_a)
{
uint64_t const max_depth (config.max_pruning_depth != 0 ? config.max_pruning_depth : std::numeric_limits<uint64_t>::max ());
uint64_t const cutoff_time (bootstrap_weight_reached_a ? nano::seconds_since_epoch () - config.max_pruning_age.count () : std::numeric_limits<uint64_t>::max ());
uint64_t pruned_count (0);
uint64_t transaction_write_count (0);
nano::account last_account (1); // 0 Burn account is never opened. So it can be used to break loop
std::deque<nano::block_hash> pruning_targets;
bool target_finished (false);
while ((transaction_write_count != 0 || !target_finished) && !stopped)
{
// Search pruning targets
while (pruning_targets.size () < batch_size_a && !target_finished && !stopped)
{
target_finished = collect_ledger_pruning_targets (pruning_targets, last_account, batch_size_a * 2, max_depth, cutoff_time);
}
// Pruning write operation
transaction_write_count = 0;
if (!pruning_targets.empty () && !stopped)
{
auto scoped_write_guard = write_database_queue.wait (nano::writer::pruning);
auto write_transaction (store.tx_begin_write ({ tables::blocks, tables::pruned }));
while (!pruning_targets.empty () && transaction_write_count < batch_size_a && !stopped)
{
auto const & pruning_hash (pruning_targets.front ());
auto account_pruned_count (ledger.pruning_action (write_transaction, pruning_hash, batch_size_a));
transaction_write_count += account_pruned_count;
pruning_targets.pop_front ();
}
pruned_count += transaction_write_count;
auto log_message (boost::str (boost::format ("%1% blocks pruned") % pruned_count));
if (!log_to_cout_a)
{
logger.try_log (log_message);
}
else
{
std::cout << log_message << std::endl;
}
}
}
auto log_message (boost::str (boost::format ("Total recently pruned block count: %1%") % pruned_count));
if (!log_to_cout_a)
{
logger.always_log (log_message);
}
else
{
std::cout << log_message << std::endl;
}
}
void nano::node::ongoing_ledger_pruning ()
{
auto bootstrap_weight_reached (ledger.cache.block_count >= ledger.bootstrap_weight_max_blocks);
ledger_pruning (flags.block_processor_batch_size != 0 ? flags.block_processor_batch_size : 2 * 1024, bootstrap_weight_reached, false);
auto ledger_pruning_interval (bootstrap_weight_reached ? config.max_pruning_age : std::min (config.max_pruning_age, std::chrono::seconds (15 * 60)));
auto this_l (shared ());
alarm.add (std::chrono::steady_clock::now () + ledger_pruning_interval, [this_l]() {
this_l->worker.push_task ([this_l]() {
this_l->ongoing_ledger_pruning ();
});
});
}
int nano::node::price (nano::uint128_t const & balance_a, int amount_a)
{
debug_assert (balance_a >= amount_a * nano::Gxrb_ratio);

3
nano/node/node.hpp
View file

@ -132,6 +132,9 @@ public:
void search_pending ();
void bootstrap_wallet ();
void unchecked_cleanup ();
bool collect_ledger_pruning_targets (std::deque<nano::block_hash> &, nano::account &, uint64_t const, uint64_t const, uint64_t const);
void ledger_pruning (uint64_t const, bool, bool);
void ongoing_ledger_pruning ();
int price (nano::uint128_t const &, int);
// The default difficulty updates to base only when the first epoch_2 block is processed
uint64_t default_difficulty (nano::work_version const) const;

48
nano/rpc_test/rpc.cpp
View file

@ -5157,6 +5157,54 @@ TEST (rpc, block_info_pruning)
ASSERT_TRUE (response2.json.get<bool> ("confirmed"));
}
TEST (rpc, pruned_exists)
{
nano::system system;
auto & node0 = *system.add_node ();
nano::node_config node_config (nano::get_available_port (), system.logging);
node_config.enable_voting = false; // Remove after allowing pruned voting
nano::node_flags node_flags;
node_flags.enable_pruning = true;
auto & node1 = *add_ipc_enabled_node (system, node_config, node_flags);
auto latest (node1.latest (nano::dev_genesis_key.pub));
auto send1 (std::make_shared<nano::send_block> (latest, nano::dev_genesis_key.pub, nano::genesis_amount - nano::Gxrb_ratio, nano::dev_genesis_key.prv, nano::dev_genesis_key.pub, *node1.work_generate_blocking (latest)));
node1.process_active (send1);
auto receive1 (std::make_shared<nano::receive_block> (send1->hash (), send1->hash (), nano::dev_genesis_key.prv, nano::dev_genesis_key.pub, *node1.work_generate_blocking (send1->hash ())));
node1.process_active (receive1);
node1.block_processor.flush ();
system.wallet (0)->insert_adhoc (nano::dev_genesis_key.prv);
ASSERT_TIMELY (5s, node1.ledger.cache.cemented_count == 3 && node1.confirmation_height_processor.current ().is_zero () && node1.confirmation_height_processor.awaiting_processing_size () == 0);
// Pruning action
{
auto transaction (node1.store.tx_begin_write ());
ASSERT_EQ (1, node1.ledger.pruning_action (transaction, send1->hash (), 1));
}
scoped_io_thread_name_change scoped_thread_name_io;
nano::node_rpc_config node_rpc_config;
nano::ipc::ipc_server ipc_server (node1, node_rpc_config);
nano::rpc_config rpc_config (nano::get_available_port (), true);
rpc_config.rpc_process.ipc_port = node1.config.ipc_config.transport_tcp.port;
nano::ipc_rpc_processor ipc_rpc_processor (system.io_ctx, rpc_config);
nano::rpc rpc (system.io_ctx, rpc_config, ipc_rpc_processor);
rpc.start ();
// Pruned block
boost::property_tree::ptree request;
request.put ("action", "pruned_exists");
request.put ("hash", send1->hash ().to_string ());
test_response response (request, rpc.config.port, system.io_ctx);
ASSERT_TIMELY (5s, response.status != 0);
ASSERT_EQ (200, response.status);
ASSERT_TRUE (response.json.get<bool> ("exists"));
// Existing block with previous pruned
boost::property_tree::ptree request2;
request2.put ("action", "pruned_exists");
request2.put ("hash", receive1->hash ().to_string ());
test_response response2 (request2, rpc.config.port, system.io_ctx);
ASSERT_TIMELY (5s, response2.status != 0);
ASSERT_EQ (200, response2.status);
ASSERT_FALSE (response2.json.get<bool> ("exists"));
}
TEST (rpc, work_peers_all)
{
nano::system system;