1.19.3 (#34)

* start updating to 22w42a

* work a bit more on 22w42a

* player chat packet

* serverbound hello packet

* Update mod.rs

* add more stuff to clientbound player chat packet

* ClientboundPlayerInfoUpdatePacket

* features enabled and container closed

* serverbound chat packets

* make it compile

* 22w43a

* ServerboundChatSessionUpdatePacket

* profile_public_key isn't Option anymore

* Update bitset.rs

* joining a server works

* fix entitydatavalue

* backtraces + fix clientbound chat message

* fix some warnings and add more ecomments

* 22w44a

* generate en_us.json

* add updating guide to codegen/readme

* fix some markdown

* update list of generated things

* metadata stuff

* Replace PJS generator mod with PixLyzer (#38)

* pixlizer extractor

* start working on shape extraction

* fix generating language

* fix pixlyzer shape generation

* use empty_shape

* generate blocks and shapes

* update pixlyzer dir

* Revert "update pixlyzer dir"

This reverts commit ee9a0e7a49936dd8569c610ba9b6455895eeff71.

* fix

* fix

* Revert "fix"

This reverts commit ad12ddcb009ccc4eeb13ddef0871db1d9322ab7d.

* fix

* detect pixlyzer fail

* fix pixlyzer

* 22w45a

* gen entities

* add async-trait dep

* update codegen/readme.md

* explain when rust_log should be used

* remove some unused code

* start fixing pixlyzer issues

* fix a thing in codegen

* almost fixed

* more progress towards 1.19.3

* 1.19.3-pre2

* fixes

* revert some hardcoded property names

* Delete clientbound_player_info_packet.rs

* handle 1.19.3 player info packets

* handle playerinforemove

* start updating to 1.19.3-rc1

* optional registries work

* fix some issues with 1.19.3

chat doesn't work yet

* aaaaaaaaaaaaaaaaa

* oh

* ignore unused shapes

* uncomment generate_blocks

* fix migrate

* 1.19.3-rc2

* fix clippy warnings

* 1.19.3-rc3

* split the azalea-buf macro into separate modules

* improve Recipe in protocol

* 1.19.3

15 files changed, 365 insertions, 135 deletions

diff --git a/codegen/README.md b/codegen/README.md
index e2cb0fcd..ba5e54a3 100755
--- a/codegen/README.md
+++ b/codegen/README.md
@@ -6,14 +6,42 @@ The directory name doesn't start with `azalea-` because it's not a Rust crate.
 
 - Python 3.8+
 - Java 17+
-- Gradle
+- Maven
 
 ## Usage
 
 Generate packet:\
-`python newpacket.py [packet id] [clientbound or serverbound] \[game/handshake/login/status\]`\
-This will create a new file in the `azalea-protocol/src/packets/\[state\] directory`. You will probably have to manually fix up the auto generated code.
+`python newpacket.py [packet id] [clientbound or serverbound] [game/handshake/login/status]`\
+This will create a new file in the `azalea-protocol/src/packets/[state] directory`. You will probably have to manually fix up the auto generated code.
+
+## Updating to a new Minecraft version
+
+First, run `python migrate.py [new version]`. This will run a script that automatically updates as much as it can, including:
+- Adding, removing, and updating packets in azalea-protocol (limited)
+- Updating supported version in README.md
+- Updating the `PROTOCOL_VERSION` variable in azalea-protocol
+- Generating blocks in azalea-block
+- Generating block shapes in azalea-physics
+- Generating registries in azalea-registries
+- Updating en_us.json in azalea-language
+- Generating entity metadata structs and parsers in azalea-world
+
+If you're lucky, that's all you're going to have to do.
+Look at the diff (`git diff`) and type-check the code (`cargo check`) to make sure everything is right. In the diff, specifically look for new comments that have "TODO".
+
+If a packet is incorrect, you'll want to find it in the Minecraft source. The name of the struct should be the same or similar as the class in the vanilla source. Now, you'll have to manually write the struct for the packet. If the packet existed in the version before and it's just being updated, you can compare against that to see what was updated. Note that if a packet is particularly complicated, you may have to implement McBufReadable and McBufWritable, but most of the time the `#[derive(McBuf)]` macro will be able to generate the impls correctly. Look at other existing packets as reference if you're confused.
+
+Finally, test by making a bot join a world. Specifically, you'll want to test the things that were updated in the version. Setting the RUST_LOG environment variable to `debug` or `trace` may help you find the source of crashes (trace shows the first few hundred bytes for every packet received so it's typically more useful, but it may log more than you want).
+
+If it all works, make a pull request. If the version you updated to is a snapshot, make it a draft PR (the main branch is for release versions).
+
+## Extracting new data
+
+At the time of writing, the following data generators are used:
+
+- [Vanilla data generator](https://wiki.vg/Data_Generators)
+- [Burger](https://github.com/Pokechu22/Burger)
+- [PixLyzer](https://gitlab.bixilon.de/bixilon/pixlyzer)
+
+Some things can be obtained from multiple generators. You should prefer them by the order above (the vanilla generator is the most reliable).
 
-Migrate to a new Minecraft version:\
-`python migrate.py [new version]`\
-This updates all the packet ids in `azalea-protocol/src/packets/mod.rs` and creates all the new packets.
diff --git a/codegen/genblocks.py b/codegen/genblocks.py
index 6b802771..45e7683e 100755
--- a/codegen/genblocks.py
+++ b/codegen/genblocks.py
@@ -9,8 +9,10 @@ import lib.utils
 
 version_id = lib.code.version.get_version_id()
 
-shape_datas = lib.extract.get_generator_mod_data(
-    version_id, 'blockCollisionShapes')
+shape_datas = lib.extract.get_pixlyzer_data(
+    version_id, 'shapes')
+pixlyzer_block_datas = lib.extract.get_pixlyzer_data(
+    version_id, 'blocks')
 
 mappings = lib.download.get_mappings_for_version(version_id)
 block_states_burger = lib.extract.get_block_states_burger(version_id)
@@ -21,7 +23,7 @@ lib.code.blocks.generate_blocks(
     block_states_burger, block_states_report, ordered_blocks, mappings)
 
 lib.code.shapes.generate_block_shapes(
-    shape_datas['blocks'], shape_datas['shapes'], block_states_report, block_states_burger, mappings)
+    pixlyzer_block_datas, shape_datas['shapes'], shape_datas['aabbs'], block_states_report, block_states_burger, mappings)
 
 lib.code.utils.fmt()
 
diff --git a/codegen/genentities.py b/codegen/genentities.py
index cdd1653e..12354fe7 100644
--- a/codegen/genentities.py
+++ b/codegen/genentities.py
@@ -3,7 +3,6 @@ import lib.code.entity
 import lib.code.utils
 import lib.download
 import lib.extract
-import sys
 
 version_id = lib.code.version.get_version_id()
 
diff --git a/codegen/genlanguage.py b/codegen/genlanguage.py
new file mode 100644
index 00000000..e3e950eb
--- /dev/null
+++ b/codegen/genlanguage.py
@@ -0,0 +1,13 @@
+import lib.code.language
+import lib.code.version
+import lib.code.utils
+import lib.download
+import lib.extract
+import lib.utils
+
+version_id = lib.code.version.get_version_id()
+language = lib.extract.get_en_us_lang(version_id)
+
+lib.code.language.write_language(language)
+
+print('Done!')
diff --git a/codegen/lib/code/blocks.py b/codegen/lib/code/blocks.py
index c32a3bbc..d5fca732 100755
--- a/codegen/lib/code/blocks.py
+++ b/codegen/lib/code/blocks.py
@@ -20,7 +20,6 @@ def generate_blocks(blocks_burger: dict, blocks_report: dict, ordered_blocks: li
     new_make_block_states_macro_code = []
     new_make_block_states_macro_code.append('make_block_states! {')
 
-
     # Find properties
     properties = {}
 
@@ -84,7 +83,6 @@ def generate_blocks(blocks_burger: dict, blocks_report: dict, ordered_blocks: li
         new_make_block_states_macro_code.append(
             f'        "{property_name}" => {property_shape_code},')
 
-
     new_make_block_states_macro_code.append('    },')
 
     # Block codegen
@@ -101,7 +99,6 @@ def generate_blocks(blocks_burger: dict, blocks_report: dict, ordered_blocks: li
             if state.get('default'):
                 default_property_variants = state.get('properties', {})
 
-
         properties_code = '{'
         for property_name in list(block_data_report.get('properties', {}).keys()):
             property_burger = None
@@ -163,6 +160,7 @@ def generate_blocks(blocks_burger: dict, blocks_report: dict, ordered_blocks: li
     with open(BLOCKS_RS_DIR, 'w') as f:
         f.write('\n'.join(new_code))
 
+
 def get_property_struct_name(property: Optional[dict], block_data_burger: dict, property_variants: list[str], mappings: Mappings) -> str:
     # these are hardcoded because otherwise they cause conflicts
     # some names inspired by https://github.com/feather-rs/feather/blob/main/feather/blocks/src/generated/table.rs
@@ -184,16 +182,23 @@ def get_property_struct_name(property: Optional[dict], block_data_burger: dict,
         return 'ChestType'
     if property_variants == ['compare', 'subtract']:
         return 'ComparatorType'
+    if 'harp' in property_variants and 'didgeridoo' in property_variants:
+        return 'Sound'
 
     if property is None:
         return ''.join(map(to_camel_case, property_variants))
 
-    property_name = None
     for class_name in [block_data_burger['class']] + block_data_burger['super']:
         property_name = mappings.get_field(
             class_name, property['field_name'])
         if property_name:
             break
+    if property_name is None:
+        if 'declared_in' in property:
+            property_name = mappings.get_field(
+                property['declared_in'], property['field_name'])
+    if property_name is None:
+        property_name = property['name']
     assert property_name
     property_name = to_camel_case(property_name.lower())
     if property['type'] == 'int':
diff --git a/codegen/lib/code/entity.py b/codegen/lib/code/entity.py
index 9b976ef6..6616c8d7 100644
--- a/codegen/lib/code/entity.py
+++ b/codegen/lib/code/entity.py
@@ -11,7 +11,8 @@ def generate_entity_metadata(burger_entity_data: dict, mappings: Mappings):
     # TODO: auto generate this and use it for generating the EntityDataValue enum
     metadata_types = [
         {'name': 'Byte', 'type': 'u8'},
-        {'name': 'Int', 'type': 'i32'},
+        {'name': 'Int', 'type': 'i32', 'var': True},
+        {'name': 'Long', 'type': 'i64'},
         {'name': 'Float', 'type': 'f32'},
         {'name': 'String', 'type': 'String'},
         {'name': 'Component', 'type': 'Component'},
@@ -23,11 +24,11 @@ def generate_entity_metadata(burger_entity_data: dict, mappings: Mappings):
         {'name': 'OptionalBlockPos', 'type': 'Option<BlockPos>'},
         {'name': 'Direction', 'type': 'Direction'},
         {'name': 'OptionalUuid', 'type': 'Option<Uuid>'},
-        {'name': 'OptionalBlockState', 'type': 'Option<BlockState>'},
+        {'name': 'BlockState', 'type': 'BlockState'},
         {'name': 'CompoundTag', 'type': 'azalea_nbt::Tag'},
         {'name': 'Particle', 'type': 'Particle'},
         {'name': 'VillagerData', 'type': 'VillagerData'},
-        {'name': 'OptionalUnsignedInt', 'type': 'Option<u32>'},
+        {'name': 'OptionalUnsignedInt', 'type': 'OptionalUnsignedInt'},
         {'name': 'Pose', 'type': 'Pose'},
         {'name': 'CatVariant', 'type': 'azalea_registry::CatVariant'},
         {'name': 'FrogVariant', 'type': 'azalea_registry::FrogVariant'},
@@ -40,7 +41,8 @@ def generate_entity_metadata(burger_entity_data: dict, mappings: Mappings):
     code.append("// Don't change it manually!")
     code.append('')
     code.append('#![allow(clippy::clone_on_copy, clippy::derivable_impls)]')
-    code.append('use super::{EntityDataValue, Rotations, VillagerData, Pose};')
+    code.append(
+        'use super::{EntityDataValue, Rotations, VillagerData, OptionalUnsignedInt, Pose};')
     code.append('use azalea_block::BlockState;')
     code.append('use azalea_chat::Component;')
     code.append('use azalea_core::{BlockPos, Direction, Particle, Slot};')
@@ -185,6 +187,8 @@ def generate_entity_metadata(burger_entity_data: dict, mappings: Mappings):
                         default = 'azalea_registry::PaintingVariant::Kebab'
                     elif type_name == 'FrogVariant':
                         default = 'azalea_registry::FrogVariant::Temperate'
+                    elif type_name == 'VillagerData':
+                        default = 'VillagerData { kind: azalea_registry::VillagerType::Plains, profession: azalea_registry::VillagerProfession::None, level: 0 }'
                     else:
                         default = 'Default::default()'
                 else:
@@ -200,11 +204,11 @@ def generate_entity_metadata(burger_entity_data: dict, mappings: Mappings):
                     elif type_name == 'OptionalUuid':
                         default = f'Some(uuid::uuid!({default}))' if default != 'Empty' else 'None'
                     elif type_name == 'OptionalUnsignedInt':
-                        default = f'Some({default})' if default != 'Empty' else 'None'
+                        default = f'OptionalUnsignedInt(Some({default}))' if default != 'Empty' else 'OptionalUnsignedInt(None)'
                     elif type_name == 'ItemStack':
                         default = f'Slot::Present({default})' if default != 'Empty' else 'Slot::Empty'
-                    elif type_name == 'OptionalBlockState':
-                        default = f'Some({default})' if default != 'Empty' else 'None'
+                    elif type_name == 'BlockState':
+                        default = f'{default}' if default != 'Empty' else 'BlockState::Air'
                     elif type_name == 'OptionalComponent':
                         default = f'Some({default})' if default != 'Empty' else 'None'
                     elif type_name == 'CompoundTag':
diff --git a/codegen/lib/code/language.py b/codegen/lib/code/language.py
new file mode 100644
index 00000000..d27dde81
--- /dev/null
+++ b/codegen/lib/code/language.py
@@ -0,0 +1,8 @@
+from lib.utils import get_dir_location
+import json
+
+LANGUAGE_DIR = get_dir_location('../azalea-language/src/en_us.json')
+
+def write_language(contents: dict):
+    with open(LANGUAGE_DIR, 'w') as f:
+        f.write(json.dumps(contents, indent='  '))
\ No newline at end of file
diff --git a/codegen/lib/code/packet.py b/codegen/lib/code/packet.py
index 4beccd35..48572aed 100755
--- a/codegen/lib/code/packet.py
+++ b/codegen/lib/code/packet.py
@@ -247,9 +247,10 @@ def burger_instruction_to_code(instructions: list[dict], index: int, generated_p
         # figure out what kind of iterator it is
         loop_instructions = next_next_instruction['instructions']
         if len(loop_instructions) == 2:
-            entry_type_rs, is_var, uses, extra_code = burger_type_to_rust_type(
+            entry_type_rs, is_var, value_uses, extra_code = burger_type_to_rust_type(
                 loop_instructions[1]['type'], None, loop_instructions[1], mappings, obfuscated_class_name)
             field_type_rs = f'Vec<{entry_type_rs}>'
+            uses.update(value_uses)
         elif len(loop_instructions) == 3:
             is_map = loop_instructions[0]['type'].startswith(
                 'Map.Entry<')
diff --git a/codegen/lib/code/registry.py b/codegen/lib/code/registry.py
index c22eefe9..4df82c09 100755
--- a/codegen/lib/code/registry.py
+++ b/codegen/lib/code/registry.py
@@ -9,8 +9,45 @@ REGISTRIES_DIR = get_dir_location('../azalea-registry/src/lib.rs')
 def generate_registries(registries: dict):
     code = []
 
-    code.append('use azalea_registry_macros::registry;')
-    code.append('')
+    code.append('''// This file is automatically generated in codegen/lib/code/registry.py
+
+use azalea_buf::{BufReadError, McBufReadable, McBufVarReadable, McBufVarWritable, McBufWritable};
+use azalea_registry_macros::registry;
+use std::io::{Cursor, Write};
+
+pub trait Registry
+where
+    Self: Sized,
+{
+    fn from_u32(value: u32) -> Option<Self>;
+    fn to_u32(&self) -> u32;
+}
+
+/// A registry that might not be present. This is transmitted as a single
+/// varint in the protocol.
+#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
+pub struct OptionalRegistry<T: Registry>(Option<T>);
+
+impl<T: Registry> McBufReadable for OptionalRegistry<T> {
+    fn read_from(buf: &mut Cursor<&[u8]>) -> Result<Self, BufReadError> {
+        Ok(OptionalRegistry(match u32::var_read_from(buf)? {
+            0 => None,
+            value => Some(
+                T::from_u32(value - 1)
+                    .ok_or(BufReadError::UnexpectedEnumVariant { id: value as i32 })?,
+            ),
+        }))
+    }
+}
+impl<T: Registry> McBufWritable for OptionalRegistry<T> {
+    fn write_into(&self, buf: &mut impl Write) -> Result<(), std::io::Error> {
+        match &self.0 {
+            None => 0u32.var_write_into(buf),
+            Some(value) => (value.to_u32() + 1).var_write_into(buf),
+        }
+    }
+}
+''')
 
     for registry_name, registry in registries.items():
         # registry!(Block, {
diff --git a/codegen/lib/code/shapes.py b/codegen/lib/code/shapes.py
index 83521dac..06170552 100755
--- a/codegen/lib/code/shapes.py
+++ b/codegen/lib/code/shapes.py
@@ -7,8 +7,8 @@ COLLISION_BLOCKS_RS_DIR = get_dir_location(
     '../azalea-physics/src/collision/blocks.rs')
 
 
-def generate_block_shapes(blocks: dict, shapes: dict, block_states_report, block_datas_burger, mappings: Mappings):
-    blocks, shapes = simplify_shapes(blocks, shapes)
+def generate_block_shapes(blocks: dict, shapes: dict, aabbs: dict, block_states_report, block_datas_burger, mappings: Mappings):
+    blocks, shapes = simplify_shapes(blocks, shapes, aabbs)
 
     code = generate_block_shapes_code(
         blocks, shapes, block_states_report, block_datas_burger, mappings)
@@ -16,33 +16,54 @@ def generate_block_shapes(blocks: dict, shapes: dict, block_states_report, block
         f.write(code)
 
 
-def simplify_shapes(blocks: dict, shapes: dict):
-    shape_to_new_id = {}
+def simplify_shapes(blocks: dict, shapes: dict, aabbs: dict):
     new_id_increment = 0
 
     new_shapes = {}
     old_id_to_new_id = {}
 
-    for shape_id, shape in sorted(shapes.items(), key=lambda shape: int(shape[0])):
-        # tuples are hashable
-        shape_as_tuple = tuple(map(tuple, shape))
-        if shape_as_tuple not in shape_to_new_id:
-            shape_to_new_id[shape_as_tuple] = new_id_increment
-            old_id_to_new_id[shape_id] = new_id_increment
-            new_shapes[new_id_increment] = shape
-            new_id_increment += 1
-        else:
-            old_id_to_new_id[shape_id] = shape_to_new_id[shape_as_tuple]
+    old_id_to_new_id[None] = 0
+    new_shapes[0] = ()
+    new_id_increment += 1
+
+    used_shape_ids = set()
+    # determine the used shape ids
+    for block_id, block_data in blocks.items():
+        block_id = block_id.split(':')[-1]
+        block_shapes = [state.get('collision_shape')
+                        for state in block_data['states'].values()]
+        for s in block_shapes:
+            used_shape_ids.add(s)
+
+    for shape_id, shape in enumerate(shapes):
+        if shape_id not in used_shape_ids: continue
+        # pixlyzer gives us shapes as an index or list of indexes into the
+        # aabbs list
+        # and aabbs look like { "from": number or [x, y, z], "to": (number or vec3) }
+        # convert them to [x1, y1, z1, x2, y2, z2]
+        shape = [shape] if isinstance(shape, int) else shape
+        shape = [aabbs[shape_aabb] for shape_aabb in shape]
+        shape = tuple([(
+            (tuple(part['from']) if isinstance(
+                part['from'], list) else ((part['from'],)*3))
+            + (tuple(part['to']) if isinstance(part['to'], list)
+               else ((part['to'],)*3))
+        ) for part in shape])
+
+        old_id_to_new_id[shape_id] = new_id_increment
+        new_shapes[new_id_increment] = shape
+        new_id_increment += 1
 
     # now map the blocks to the new shape ids
-    for block_id, shape_ids in blocks.items():
-        if isinstance(shape_ids, int):
-            blocks[block_id] = old_id_to_new_id[str(shape_ids)]
-        else:
-            blocks[block_id] = [old_id_to_new_id[str(shape_id)]
-                                for shape_id in shape_ids]
+    new_blocks = {}
+    for block_id, block_data in blocks.items():
+        block_id = block_id.split(':')[-1]
+        block_shapes = [state.get('collision_shape')
+                        for state in block_data['states'].values()]
+        new_blocks[block_id] = [old_id_to_new_id[shape_id]
+                                for shape_id in block_shapes]
 
-    return blocks, new_shapes
+    return new_blocks, new_shapes
 
 
 def generate_block_shapes_code(blocks: dict, shapes: dict, block_states_report, block_datas_burger, mappings: Mappings):
@@ -116,7 +137,7 @@ def generate_code_for_shape(shape_id: str, parts: list[list[float]]):
     code = ''
     code += f'static SHAPE{shape_id}: Lazy<VoxelShape> = Lazy::new(|| {{'
     steps = []
-    if parts == []:
+    if parts == ():
         steps.append('collision::empty_shape()')
     else:
         steps.append(f'collision::box_shape({make_arguments(parts[0])})')
diff --git a/codegen/lib/code/utils.py b/codegen/lib/code/utils.py
index 66b18eed..5550cdb2 100755
--- a/codegen/lib/code/utils.py
+++ b/codegen/lib/code/utils.py
@@ -63,6 +63,12 @@ def burger_type_to_rust_type(burger_type, field_name: Optional[str] = None, inst
     elif burger_type == 'metadata':
         field_type_rs = 'EntityMetadata'
         uses.add('azalea_entity::EntityMetadata')
+    elif burger_type == 'bitset':
+        if instruction:
+            length = instruction['length']
+            field_type_rs = f'todo!("fixed bitset of length {length}")'
+        else:
+            field_type_rs = 'todo!("fixed bitset")'
     elif burger_type == 'abstract':
         field_type_rs = 'todo!()'
     elif burger_type == 'enum':
diff --git a/codegen/lib/download.py b/codegen/lib/download.py
index 5988f8ec..aec3a648 100755
--- a/codegen/lib/download.py
+++ b/codegen/lib/download.py
@@ -19,15 +19,15 @@ def get_burger():
             f'cd {get_dir_location("downloads")} && git clone https://github.com/pokechu22/Burger && cd Burger && git pull')
 
         print('\033[92mInstalling dependencies...\033[m')
-        os.system('cd downloads/Burger && pip install six jawa')
+        os.system(f'cd {get_dir_location("downloads")}/Burger && pip install six jawa')
 
 
-def get_generator_mod():
-    if not os.path.exists(get_dir_location('downloads/minecraft-data-generator-server')):
-        print('\033[92mDownloading u9g/minecraft-data-generator-server...\033[m')
+def get_pixlyzer():
+    if not os.path.exists(get_dir_location('downloads/pixlyzer')):
+        print('\033[92mDownloading bixilon/pixlyzer...\033[m')
         os.system(
-            f'cd {get_dir_location("downloads")} && git clone https://github.com/u9g/minecraft-data-generator-server && cd minecraft-data-generator-server && git pull')
-    return get_dir_location('downloads/minecraft-data-generator-server')
+            f'cd {get_dir_location("downloads")} && git clone https://gitlab.bixilon.de/bixilon/pixlyzer.git && cd pixlyzer && git pull')
+    return get_dir_location('downloads/pixlyzer')
 
 
 def get_version_manifest():
@@ -180,4 +180,4 @@ def clear_version_cache():
     os.system(
         f'cd {get_dir_location("downloads/Burger")} && git pull')
     os.system(
-        f'cd {get_dir_location("downloads/minecraft-data-generator-server")} && git pull')
+        f'cd {get_dir_location("downloads/pixlyzer")} && git pull')
diff --git a/codegen/lib/extract.py b/codegen/lib/extract.py
index e66b9400..7cbeff30 100755
--- a/codegen/lib/extract.py
+++ b/codegen/lib/extract.py
@@ -1,9 +1,11 @@
 # Extracting data from the Minecraft jars
 
-from lib.download import get_server_jar, get_burger, get_client_jar, get_generator_mod, get_yarn_data, get_fabric_api_versions, get_fabric_loader_versions
+from lib.download import get_server_jar, get_burger, get_client_jar, get_pixlyzer, get_yarn_data, get_fabric_api_versions, get_fabric_loader_versions
 from lib.utils import get_dir_location
+from zipfile import ZipFile
 import subprocess
 import json
+import sys
 import re
 import os
 
@@ -59,103 +61,185 @@ def determine_python_command():
         'Couldn\'t determine python command to use to run burger with!')
 
 
+def run_python_command_and_download_deps(command):
+    print('>', command)
+    for _ in range(10):
+        p = subprocess.Popen(
+            command,
+            stderr=subprocess.PIPE,
+            shell=True
+        )
+
+        stderr = b''
+        while True:
+            data = p.stderr.read()
+            if data == b'':
+                break
+            print(data.decode(), end='', flush=True)
+            stderr += data
+
+        regex_match = re.search(
+            r'ModuleNotFoundError: No module named \'(\w+?)\'', stderr.decode())
+        if not regex_match:
+            out, err = p.communicate()
+            if out:
+                print(out)
+            if err:
+                print(err)
+            break
+        missing_lib = regex_match.group(1)
+        print('Missing required lib:', missing_lib)
+        os.system(
+            f'{determine_python_command()} -m pip install {missing_lib}')
+    print('ok')
+
+
 def get_burger_data_for_version(version_id: str):
     if not os.path.exists(get_dir_location(f'downloads/burger-{version_id}.json')):
         get_burger()
         get_client_jar(version_id)
 
-        for _ in range(10):
-            r = subprocess.run(
-                f'cd {get_dir_location("downloads/Burger")} && {determine_python_command()} munch.py ../client-{version_id}.jar --output ../burger-{version_id}.json',
-                capture_output=True,
-                shell=True
-            )
-            regex_match = re.search(
-                r'ModuleNotFoundError: No module named \'(\w+?)\'', r.stderr.decode())
-            if not regex_match:
-                break
-            missing_lib = regex_match.group(1)
-            print('Missing required lib for Burger:', missing_lib)
-            os.system(
-                f'{determine_python_command()} -m pip install {missing_lib}')
+        print('\033[92mRunning Burger...\033[m')
+        run_python_command_and_download_deps(
+            f'cd {get_dir_location("downloads/Burger")} && {determine_python_command()} munch.py {get_dir_location("downloads")}/client-{version_id}.jar --output {get_dir_location("downloads")}/burger-{version_id}.json'
+        )
     with open(get_dir_location(f'downloads/burger-{version_id}.json'), 'r') as f:
         return json.load(f)
 
 
-def get_generator_mod_data(version_id: str, category: str):
+def get_pixlyzer_data(version_id: str, category: str):
     '''
-    Gets data from u9g's data generator mod. Note that this is not very stable, and it requires Yarn to release updates first.
+    Gets data from Pixlyzer. Note that this requires Yarn to release updates first.
     '''
 
-    target_dir = get_dir_location(f'downloads/generator-mod-{version_id}')
+    target_dir = get_dir_location(f'downloads/pixlyzer-{version_id}')
 
     if not os.path.exists(get_dir_location(target_dir)):
-        generator_mod_dir = get_generator_mod()
-
-        yarn_data = get_yarn_data(version_id)
-        if not yarn_data:
-            raise Exception(
-                'Fabric/Yarn hasn\'t been updated to this version yet.')
-        # looks like 1.19+build.1
-        yarn_version = yarn_data['version']
-
-        fabric_api_version = get_fabric_api_versions()[-1]
-        fabric_loader_version = get_fabric_loader_versions()[0]
-
-        # the mod has the minecraft version hard-coded by default, so we just change the gradle.properties and fabric.mod.json
-        with open(get_dir_location(f'{generator_mod_dir}/gradle.properties'), 'r') as f:
-            lines = f.readlines()
-        with open(get_dir_location(f'{generator_mod_dir}/gradle.properties'), 'w') as f:
-            for line in lines:
-                if line.startswith('minecraft_version='):
-                    line = f'minecraft_version={version_id}\n'
-                if line.startswith('yarn_mappings='):
-                    line = f'yarn_mappings={yarn_version}\n'
-                if line.startswith('fabric_version='):
-                    line = f'fabric_version={fabric_api_version}\n'
-                if line.startswith('loader_version='):
-                    line = f'loader_version={fabric_loader_version}\n'
-                f.write(line)
-        # edit the fabric.mod.json to support this version
-        with open(get_dir_location(f'{generator_mod_dir}/src/main/resources/fabric.mod.json'), 'r') as f:
-            fabric_mod_json = json.load(f)
-        fabric_mod_json['depends']['minecraft'] = '*'
-        with open(get_dir_location(f'{generator_mod_dir}/src/main/resources/fabric.mod.json'), 'w') as f:
-            json.dump(fabric_mod_json, f, indent=2)
-
-        try:
-            os.system(f'cd {generator_mod_dir} && chmod u+x ./gradlew')
-        except:
-            pass
-
-        # set the server port to something other than 25565 so it doesn't
-        # conflict with anything else that's running
-        try:
-            os.makedirs(get_dir_location(f'{generator_mod_dir}/run'))
-        except:
-            pass
-        with open(get_dir_location(f'{generator_mod_dir}/run/server.properties'), 'w') as f:
-            f.write('server-port=56553')
-
-        # make sure we have perms to run this file
-        # (on windows it fails but keeps running)
-        os.system(f'cd {generator_mod_dir} && chmod u+x ./gradlew')
-        try:
-            subprocess.run(
-                [f'cd {generator_mod_dir} && ./gradlew runServer'],
-                check=True,
-                shell=True
-            )
-        except Exception as e:
-            os.system(f'cd {generator_mod_dir} && gradlew runServer')
+        pixlyzer_dir = get_pixlyzer()
 
+        # for some reason pixlyzer doesn't work right unless the mvn clean
+        # instruction looks like that
+        # and pixlyzer.py doesn't do it right
+
+        # map jar + download dependencies
+        run_python_command_and_download_deps(
+            f'cd {pixlyzer_dir}/wrapper && {determine_python_command()} PixLyzer.py --only-version={version_id} --dont-compile --only-map'
+        )
+        # update the pom.xml <dependencies>
+        # list directories in pixlyzer/wrapper/data/data/dependencies/libraries
+        pom_xml_dependencies = '''<dependency>
+            <groupId>org.jetbrains.kotlin</groupId>
+            <artifactId>kotlin-test-junit</artifactId>
+            <version>1.7.21</version>
+            <scope>test</scope>
+        </dependency>
+        <dependency>
+            <groupId>org.jetbrains.kotlin</groupId>
+            <artifactId>kotlin-stdlib-jdk8</artifactId>
+            <version>1.7.21</version>
+        </dependency>
+
+        <dependency>
+            <groupId>net.minecraft</groupId>
+            <artifactId>client</artifactId>
+            <version>${minecraft.version}</version>
+            <scope>system</scope>
+            <systemPath>${project.basedir}/wrapper/data/data/${minecraft.version}_yarn/${minecraft.version}-exhibitionism.jar</systemPath>
+        </dependency>
+        <dependency>
+            <groupId>de.bixilon</groupId>
+            <artifactId>mbf-kotlin</artifactId>
+            <version>0.2.1</version>
+        </dependency>
+        <dependency>
+            <groupId>org.objenesis</groupId>
+            <artifactId>objenesis</artifactId>
+            <version>3.3</version>
+        </dependency>
+        <dependency>
+            <groupId>org.apache.commons</groupId>
+            <artifactId>commons-lang3</artifactId>
+            <version>3.12.0</version>
+        </dependency>
+        <dependency>
+            <groupId>com.fasterxml.jackson.core</groupId>
+            <artifactId>jackson-databind</artifactId>
+            <version>2.14.0</version>
+        </dependency>
+        <dependency>
+            <groupId>de.bixilon</groupId>
+            <artifactId>kutil</artifactId>
+            <version>1.17.1</version>
+        </dependency>'''
+        # walk dir f'{pixlyzer_dir}/wrapper/data/data/dependencies/libraries'
+        for root, dirs, files in os.walk(f'{pixlyzer_dir}/wrapper/data/data/dependencies/libraries'):
+            for file in files:
+                full_path = os.path.join(
+                    root.replace('\\', '/').replace(
+                        f'{pixlyzer_dir}/wrapper/data/data/dependencies/libraries/'.replace('\\', '/'), ''),
+                    file
+                ).replace('\\', '/')
+                print(full_path)
+                if not full_path.endswith('.jar'):
+                    continue
+                split_path = full_path.split('/')
+                group = ''
+                for group_index in range(0, len(split_path) - 3):
+                    group += split_path[group_index] + '.'
+                if group.endswith('.'):
+                    group = group[:-1]
+                artifact = split_path[-3]
+                version = split_path[-2]
+                path = '${project.basedir}/wrapper/data/data/dependencies/libraries/' + full_path
+                pom_xml_dependencies += """
+                    <dependency>
+                        <groupId>""" + group + """</groupId>
+                        <artifactId>""" + artifact + """</artifactId>
+                        <version>""" + version + """</version>
+                        <scope>system</scope>
+                        <systemPath>""" + path + """</systemPath>
+                    </dependency>
+                    """
+        print('pom_xml_dependencies', pom_xml_dependencies)
+        assert pom_xml_dependencies != ''
+        pom_xml = open(f'{pixlyzer_dir}/pom.xml', 'r').read()
+        pom_xml = re.sub(
+            '<dependencies>.*?</dependencies>', f'<dependencies>{pom_xml_dependencies}</dependencies>', pom_xml, flags=re.DOTALL)
+        open(f'{pixlyzer_dir}/pom.xml', 'w').write(pom_xml)
+
+        # compile
+        os.system(
+            f'cd {pixlyzer_dir} && mvn clean -Dmaven.repo.local=. verify')
+        # run pixlyzer.py again lol
+        run_python_command_and_download_deps(
+            f'cd {pixlyzer_dir}/wrapper && {determine_python_command()} PixLyzer.py --only-version={version_id} --no-compile'
+        )
+
+        source_dir = get_dir_location(
+            f'{pixlyzer_dir}/wrapper/data/version/{version_id}')
+
+        if not os.path.exists(source_dir):
+            print('PixLyzer failed, no output!')
+            exit()
         if os.path.exists(target_dir):
             os.unlink(target_dir)
         os.rename(
-            get_dir_location(
-                f'{generator_mod_dir}/run/minecraft-data/{version_id}'),
+            source_dir,
             target_dir
         )
 
-    with open(f'{target_dir}/{category}.json', 'r') as f:
+    with open(f'{target_dir}/{category}.min.json', 'r') as f:
         return json.load(f)
+
+
+def get_file_from_jar(version_id: str, file_dir: str):
+    get_client_jar(version_id)
+    with ZipFile(get_dir_location(f'downloads/client-{version_id}.jar')) as z:
+        with z.open(file_dir) as f:
+            return f.read()
+
+
+def get_en_us_lang(version_id: str):
+    return json.loads(
+        get_file_from_jar(version_id, 'assets/minecraft/lang/en_us.json')
+    )
diff --git a/codegen/migrate.py b/codegen/migrate.py
index 50bdb354..9df3ee0d 100755
--- a/codegen/migrate.py
+++ b/codegen/migrate.py
@@ -1,8 +1,11 @@
 from lib.code.packet import fix_state
 from lib.utils import PacketIdentifier, group_packets
+import lib.code.language
+import lib.code.registry
 import lib.code.version
 import lib.code.blocks
 import lib.code.packet
+import lib.code.shapes
 import lib.code.utils
 import lib.download
 import lib.extract
@@ -101,7 +104,6 @@ for packet in added_or_changed_packets:
 
 lib.code.version.set_protocol_version(
     new_burger_data[0]['version']['protocol'])
-lib.code.version.set_version_id(new_version_id)
 
 print('Updated protocol!')
 
@@ -110,13 +112,33 @@ old_ordered_blocks = lib.extract.get_ordered_blocks_burger(old_version_id)
 new_ordered_blocks = lib.extract.get_ordered_blocks_burger(new_version_id)
 if old_ordered_blocks != new_ordered_blocks:
     print('Blocks changed, updating...')
+
     block_states_burger = lib.extract.get_block_states_burger(new_version_id)
     block_states_report = lib.extract.get_block_states_report(new_version_id)
 
+    shape_datas = lib.extract.get_pixlyzer_data(
+        new_version_id, 'shapes')
+    pixlyzer_block_datas = lib.extract.get_pixlyzer_data(
+        new_version_id, 'blocks')
+
     lib.code.blocks.generate_blocks(
-        block_states_burger, block_states_report, old_ordered_blocks, new_mappings)
+        block_states_burger, block_states_report, new_ordered_blocks, new_mappings)
+    lib.code.shapes.generate_block_shapes(
+        pixlyzer_block_datas, shape_datas['shapes'], shape_datas['aabbs'], block_states_report, block_states_burger, new_mappings)
+
+print('Getting en_us.json...')
+language = lib.extract.get_en_us_lang(new_version_id)
+lib.code.language.write_language(language)
+
+print('Generating registries...')
+registries = lib.extract.get_registries_report(new_version_id)
+lib.code.registry.generate_registries(registries)
+
+print('Finishing touches, setting version in README and formatting code...')
+lib.code.version.set_version_id(new_version_id)
 
 
 lib.code.utils.fmt()
 
 print('Done!')
+print('Make sure to `cargo check` and look for the generated `TODO`s to make sure everything is correct!')
diff --git a/codegen/newpacket.py b/codegen/newpacket.py
index 48d97640..d8edcde1 100755
--- a/codegen/newpacket.py
+++ b/codegen/newpacket.py
@@ -11,7 +11,7 @@ mappings = lib.download.get_mappings_for_version(version_id)
 burger_data = lib.extract.get_burger_data_for_version(version_id)
 
 burger_packets_data = burger_data[0]['packets']['packet']
-packet_id, direction, state = int(sys.argv[1]), sys.argv[2], sys.argv[3]
+packet_id, direction, state = int(sys.argv[1], 0), sys.argv[2], sys.argv[3]
 print(
     f'Generating code for packet id: {packet_id} with direction {direction} and state {state}')
 lib.code.packet.generate_packet(burger_packets_data, mappings,