Docker Setup

I have setup the docker container to run the model evaluation.

Initial Query

Write a self-contained Rust program that prints a JSON output of how often each word appears in a text file located in /tmp/file.txt. So if the file contains "This is is is, an example" then the program should print to stdout. {"This": 1, "is": 2, "is,": 1, "an": 1, "example": 1} Don't import any crates that would require a Cargo.toml file. I'll just compile it with `rustc yourfile.rs` and it should work. The program should be run by calling /path/to/rust_word_count with no arguments. Make it run in parallel on 4 cores at the same time, each job taking ~1/4 of the file. Print out 4 differnt json dicts, one per line. I'll do the merging.

LLM Generation

Query

Write a self-contained Rust program that prints a JSON output of how often each word appears in a text file located in /tmp/file.txt. So if the file contains "This is is is, an example" then the program should print to stdout. {"This": 1, "is": 2, "is,": 1, "an": 1, "example": 1} Don't import any crates that would require a Cargo.toml file. I'll just compile it with `rustc yourfile.rs` and it should work. The program should be run by calling /path/to/rust_word_count with no arguments. Make it run in parallel on 4 cores at the same time, each job taking ~1/4 of the file. Print out 4 differnt json dicts, one per line. I'll do the merging.

Output

Below is one complete Rust program that uses only the standard library. It reads the whole file, splits it into words (using split_whitespace), partitions the list of words into four (approximately equal) pieces, spawns four threads (one per core) that each count the words in its chunk, and then prints a JSON‐formatted dictionary (one per line). You can compile with   rustc yourfile.rs and run with no arguments. -------------------------------------------------- #!/usr/bin/env rust use std::collections::HashMap; use std::fs; use std::thread; fn main() { // Read the entire file from /tmp/file.txt. let contents = fs::read_to_string("/tmp/file.txt") .expect("Failed to read /tmp/file.txt"); // Split file contents into words. // Here “word” means any sequence separated by whitespace. let words: Vec<&str> = contents.split_whitespace().collect(); // Determine the chunk sizes. (Last chunk gets any extra words.) let total_words = words.len(); let base_chunk = total_words / 4; let mut chunks = Vec::with_capacity(4); let mut start = 0; for i in 0..4 { // Let the last chunk take the remainder. let end = if i == 3 { total_words } else { start + base_chunk }; chunks.push(&words[start..end]); start = end; } // Spawn 4 threads – each one processes approximately 1/4 of the words. let mut handles = Vec::new(); for chunk in chunks { // Clone or capture the slice into the thread. // We move the slice into the thread via a new variable. let chunk = chunk.to_owned(); let handle = thread::spawn(move || -> String { let mut counts = HashMap::new(); // Count occurrences of each word. for word in chunk { *counts.entry(word).or_insert(0) += 1; } // Build a JSON dict string. We will format it as: // {"word": count, "..." : ...} // Note: the order of keys is arbitrary. let mut json = String::from("{"); let mut first = true; for (word, count) in counts { if !first { json.push_str(", "); } // Print the word quoted, and count as a number. // We escape any double-quote characters in the word. let escaped_word = word.replace("\"", "\\\""); json.push_str(&format!("\"{}\": {}", escaped_word, count)); first = false; } json.push('}'); json }); handles.push(handle); } // Join threads and print each resulting JSON dictionary on its own line. for handle in handles { let json_line = handle.join().expect("Thread panicked"); println!("{}", json_line); } } -------------------------------------------------- How the program works: 1. It reads the file /tmp/file.txt as a string. 2. It splits the contents on whitespace to obtain all words. (Note that punctuation is kept as part of a word – e.g. "is," remains different from "is".) 3. The word vector is then divided into 4 parts. (If the total number of words is not divisible by 4, the last thread gets the extra words.) 4. Each thread counts the words in its chunk using a HashMap. 5. The results are formatted into a JSON string (with double quotes around keys) and returned to the main thread. 6. The main thread prints out 4 JSON dictionaries (one per line). This program meets the requirements and uses only the standard library so no Cargo.toml is needed.

Extract Code

I extracted the following code from that output: 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
#!/usr/bin/env rust
use std::collections::HashMap;
use std::fs;
use std::thread;

fn main() {
    // Read the entire file from /tmp/file.txt.
    let contents = fs::read_to_string("/tmp/file.txt")
        .expect("Failed to read /tmp/file.txt");

    // Split file contents into words.
    // Here “word” means any sequence separated by whitespace.
    let words: Vec<&str> = contents.split_whitespace().collect();

    // Determine the chunk sizes. (Last chunk gets any extra words.)
    let total_words = words.len();
    let base_chunk = total_words / 4;
    let mut chunks = Vec::with_capacity(4);
    let mut start = 0;
    for i in 0..4 {
        // Let the last chunk take the remainder.
        let end = if i == 3 {
            total_words
        } else {
            start + base_chunk
        };
        chunks.push(&words[start..end]);
        start = end;
    }

    // Spawn 4 threads – each one processes approximately 1/4 of the words.
    let mut handles = Vec::new();
    for chunk in chunks {
        // Clone or capture the slice into the thread.
        // We move the slice into the thread via a new variable.
        let chunk = chunk.to_owned();
        let handle = thread::spawn(move || -> String {
            let mut counts = HashMap::new();
            // Count occurrences of each word.
            for word in chunk {
                *counts.entry(word).or_insert(0) += 1;
            }
            // Build a JSON dict string. We will format it as:
            // {"word": count, "..." : ...}
            // Note: the order of keys is arbitrary.
            let mut json = String::from("{");
            let mut first = true;
            for (word, count) in counts {
                if !first {
                    json.push_str(", ");
                }
                // Print the word quoted, and count as a number.
                // We escape any double-quote characters in the word.
                let escaped_word = word.replace("\"", "\\\"");
                json.push_str(&format!("\"{}\": {}", escaped_word, count));
                first = false;
            }
            json.push('}');
            json
        });
        handles.push(handle);
    }

    // Join threads and print each resulting JSON dictionary on its own line.
    for handle in handles {
        let json_line = handle.join().expect("Thread panicked");
        println!("{}", json_line);
    }
}

Check if all of the following conditions are true:

LLM Generation

Query

Below is some student's code. I asked them to parallelize some code. Do not judge if it is correct, I have test cases for that. Logic errors are okay. So are synatx errors. Just tell me if they tried to parallelize it. #!/usr/bin/env rust use std::collections::HashMap; use std::fs; use std::thread; fn main() { // Read the entire file from /tmp/file.txt. let contents = fs::read_to_string("/tmp/file.txt") .expect("Failed to read /tmp/file.txt"); // Split file contents into words. // Here “word” means any sequence separated by whitespace. let words: Vec<&str> = contents.split_whitespace().collect(); // Determine the chunk sizes. (Last chunk gets any extra words.) let total_words = words.len(); let base_chunk = total_words / 4; let mut chunks = Vec::with_capacity(4); let mut start = 0; for i in 0..4 { // Let the last chunk take the remainder. let end = if i == 3 { total_words } else { start + base_chunk }; chunks.push(&words[start..end]); start = end; } // Spawn 4 threads – each one processes approximately 1/4 of the words. let mut handles = Vec::new(); for chunk in chunks { // Clone or capture the slice into the thread. // We move the slice into the thread via a new variable. let chunk = chunk.to_owned(); let handle = thread::spawn(move || -> String { let mut counts = HashMap::new(); // Count occurrences of each word. for word in chunk { *counts.entry(word).or_insert(0) += 1; } // Build a JSON dict string. We will format it as: // {"word": count, "..." : ...} // Note: the order of keys is arbitrary. let mut json = String::from("{"); let mut first = true; for (word, count) in counts { if !first { json.push_str(", "); } // Print the word quoted, and count as a number. // We escape any double-quote characters in the word. let escaped_word = word.replace("\"", "\\\""); json.push_str(&format!("\"{}\": {}", escaped_word, count)); first = false; } json.push('}'); json }); handles.push(handle); } // Join threads and print each resulting JSON dictionary on its own line. for handle in handles { let json_line = handle.join().expect("Thread panicked"); println!("{}", json_line); } } Answer either 'The student passes' or 'The student fails'.

Output

The student passes.

Substring Evaluation

Testing if the previous output contains the string student passes: True

Run Code Interpreter

Running the following program: 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
#!/usr/bin/env rust
use std::collections::HashMap;
use std::fs;
use std::thread;

fn main() {
    // Read the entire file from /tmp/file.txt.
    let contents = fs::read_to_string("/tmp/file.txt")
        .expect("Failed to read /tmp/file.txt");

    // Split file contents into words.
    // Here “word” means any sequence separated by whitespace.
    let words: Vec<&str> = contents.split_whitespace().collect();

    // Determine the chunk sizes. (Last chunk gets any extra words.)
    let total_words = words.len();
    let base_chunk = total_words / 4;
    let mut chunks = Vec::with_capacity(4);
    let mut start = 0;
    for i in 0..4 {
        // Let the last chunk take the remainder.
        let end = if i == 3 {
            total_words
        } else {
            start + base_chunk
        };
        chunks.push(&words[start..end]);
        start = end;
    }

    // Spawn 4 threads – each one processes approximately 1/4 of the words.
    let mut handles = Vec::new();
    for chunk in chunks {
        // Clone or capture the slice into the thread.
        // We move the slice into the thread via a new variable.
        let chunk = chunk.to_owned();
        let handle = thread::spawn(move || -> String {
            let mut counts = HashMap::new();
            // Count occurrences of each word.
            for word in chunk {
                *counts.entry(word).or_insert(0) += 1;
            }
            // Build a JSON dict string. We will format it as:
            // {"word": count, "..." : ...}
            // Note: the order of keys is arbitrary.
            let mut json = String::from("{");
            let mut first = true;
            for (word, count) in counts {
                if !first {
                    json.push_str(", ");
                }
                // Print the word quoted, and count as a number.
                // We escape any double-quote characters in the word.
                let escaped_word = word.replace("\"", "\\\"");
                json.push_str(&format!("\"{}\": {}", escaped_word, count));
                first = false;
            }
            json.push('}');
            json
        });
        handles.push(handle);
    }

    // Join threads and print each resulting JSON dictionary on its own line.
    for handle in handles {
        let json_line = handle.join().expect("Thread panicked");
        println!("{}", json_line);
    }
}
And got the output: error[E0597]: `contents` does not live long enough --> main.rs:13:28 | 8 | let contents = fs::read_to_string("/tmp/file.txt") | -------- binding `contents` declared here ... 13 | let words: Vec<&str> = contents.split_whitespace().collect(); | ^^^^^^^^------------------- | | | borrowed value does not live long enough | argument requires that `contents` is borrowed for `'static` ... 69 | } | - `contents` dropped here while still borrowed error: aborting due to 1 previous error For more information about this error, try `rustc --explain E0597`. main.sh: line 2: ./a.out: No such file or directory

PyFunc

Resulting in output: []

Equal Evaluation

Testing if the previous output equals the string [('Darkness,', 1), ('Heaven,', 1), ('Light,', 1), ('age', 2), ('all', 2), ('before', 2), ('belief,', 1), ('best', 1), ('despair,', 1), ('direct', 2), ('epoch', 2), ('everything', 1), ('foolishness,', 1), ('going', 2), ('had', 2), ('hope,', 1), ('incredulity,', 1), ('it', 10), ('nothing', 1), ('of', 10), ('other', 1), ('season', 2), ('spring', 1), ('the', 11), ('times,', 2), ('to', 1), ('us,', 2), ('was', 10), ('way', 1), ('we', 4), ('were', 2), ('winter', 1), ('wisdom,', 1), ('worst', 1)]: False

Final Answer: False

Docker Setup

I have setup the docker container to run the model evaluation.

Initial Query

Write a self-contained Rust program that prints a JSON output of how often each word appears in a text file located in /tmp/file.txt. So if the file contains "This is is is, an example" then the program should print to stdout. {"This": 1, "is": 2, "is,": 1, "an": 1, "example": 1} Don't import any crates that would require a Cargo.toml file. I'll just compile it with `rustc yourfile.rs` and it should work. The program should be run by calling /path/to/rust_word_count with no arguments. Make it run in parallel on 4 cores at the same time, each job taking ~1/4 of the file. Print out 4 differnt json dicts, one per line. I'll do the merging.

LLM Generation

Query

Write a self-contained Rust program that prints a JSON output of how often each word appears in a text file located in /tmp/file.txt. So if the file contains "This is is is, an example" then the program should print to stdout. {"This": 1, "is": 2, "is,": 1, "an": 1, "example": 1} Don't import any crates that would require a Cargo.toml file. I'll just compile it with `rustc yourfile.rs` and it should work. The program should be run by calling /path/to/rust_word_count with no arguments. Make it run in parallel on 4 cores at the same time, each job taking ~1/4 of the file. Print out 4 differnt json dicts, one per line. I'll do the merging.

Output

#!/usr/bin/env rust use std::collections::HashMap; use std::fs; use std::thread; fn main() { // Read the file into a String. Panic on error. let text = fs::read_to_string("/tmp/file.txt").expect("Failed to read /tmp/file.txt"); let total_len = text.len(); if total_len == 0 { // If file is empty, then print four empty JSON dictionaries. for _ in 0..4 { println!("{{}}"); } return; } // We want to divide the text into 4 roughly equal chunks. // To avoid cutting a word in half, we adjust the boundaries: let chunk_count = 4; let chunk_size = total_len / chunk_count; let mut offsets = Vec::with_capacity(chunk_count + 1); offsets.push(0); // For chunk boundaries at indices 1..(chunk_count), adjust forward to the next whitespace. for i in 1..chunk_count { let mut idx = i * chunk_size; // Ensure idx is at a valid UTF-8 boundary. while idx < total_len && !text.is_char_boundary(idx) { idx += 1; } // Now, if we are in the middle of a word (i.e. the character before idx is not whitespace), // then move forward until we pass the current word. if idx < total_len { // Check previous char if exists. if idx > 0 { let prev_char = text[..idx].chars().rev().next().unwrap(); if !prev_char.is_whitespace() { // We are in the middle of a word, so skip characters until we hit a whitespace. while idx < total_len { // Get the next char. let c = text[idx..].chars().next().unwrap(); idx += c.len_utf8(); if c.is_whitespace() { break; } } } } } offsets.push(idx); } offsets.push(total_len); // Now we have chunk boundaries in offsets: [start0, start1, start2, start3, end]. // Spawn 4 threads, each processing one chunk. let mut handles = Vec::new(); // We want to move the Arc'ed text to threads. Because our text is owned and immutable, // we can pass slices as &str since they have the same lifetime. for i in 0..chunk_count { let chunk = &text[offsets[i]..offsets[i+1]]; // Convert the chunk string slice into an owned String so the thread is independent. // (Alternatively, we might clone the &str reference if we used Arc but here cloning the chunk slice is simple.) let chunk_owned = chunk.to_string(); let handle = thread::spawn(move || { let mut counts: HashMap<String, usize> = HashMap::new(); // Split the chunk by whitespace. This preserves punctuation attached to words. for word in chunk_owned.split_whitespace() { // This will count words exactly as they appear. *counts.entry(word.to_string()).or_insert(0) += 1; } counts }); handles.push(handle); } // Join all threads and print their JSON dictionaries, one per line. for handle in handles { let counts = handle.join().expect("Thread panicked"); // Create JSON string manually. // Note: the order of keys is arbitrary. let mut json = String::from("{"); let mut first = true; for (key, count) in counts { if !first { json.push_str(", "); } // Escape double quotes in the key if any. let escaped_key = key.replace("\"", "\\\""); json.push_str(&format!("\"{}\": {}", escaped_key, count)); first = false; } json.push('}'); println!("{}", json); } }

Extract Code

I extracted the following code from that output: 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
#!/usr/bin/env rust
use std::collections::HashMap;
use std::fs;
use std::thread;

fn main() {
    // Read the file into a String. Panic on error.
    let text = fs::read_to_string("/tmp/file.txt").expect("Failed to read /tmp/file.txt");

    let total_len = text.len();
    if total_len == 0 {
        // If file is empty, then print four empty JSON dictionaries.
        for _ in 0..4 {
            println!("{{}}");
        }
        return;
    }

    // We want to divide the text into 4 roughly equal chunks.
    // To avoid cutting a word in half, we adjust the boundaries:
    let chunk_count = 4;
    let chunk_size = total_len / chunk_count;
    let mut offsets = Vec::with_capacity(chunk_count + 1);
    offsets.push(0);

    // For chunk boundaries at indices 1..(chunk_count), adjust forward to the next whitespace.
    for i in 1..chunk_count {
        let mut idx = i * chunk_size;
        // Ensure idx is at a valid UTF-8 boundary.
        while idx < total_len && !text.is_char_boundary(idx) {
            idx += 1;
        }
        // Now, if we are in the middle of a word (i.e. the character before idx is not whitespace),
        // then move forward until we pass the current word.
        if idx < total_len {
            // Check previous char if exists.
            if idx > 0 {
                let prev_char = text[..idx].chars().rev().next().unwrap();
                if !prev_char.is_whitespace() {
                    // We are in the middle of a word, so skip characters until we hit a whitespace.
                    while idx < total_len {
                        // Get the next char.
                        let c = text[idx..].chars().next().unwrap();
                        idx += c.len_utf8();
                        if c.is_whitespace() {
                            break;
                        }
                    }
                }
            }
        }
        offsets.push(idx);
    }
    offsets.push(total_len);

    // Now we have chunk boundaries in offsets: [start0, start1, start2, start3, end].
    // Spawn 4 threads, each processing one chunk.
    let mut handles = Vec::new();
    // We want to move the Arc'ed text to threads. Because our text is owned and immutable,
    // we can pass slices as &str since they have the same lifetime.
    for i in 0..chunk_count {
        let chunk = &text[offsets[i]..offsets[i+1]];
        // Convert the chunk string slice into an owned String so the thread is independent.
        // (Alternatively, we might clone the &str reference if we used Arc but here cloning the chunk slice is simple.)
        let chunk_owned = chunk.to_string();
        let handle = thread::spawn(move || {
            let mut counts: HashMap<String, usize> = HashMap::new();
            // Split the chunk by whitespace. This preserves punctuation attached to words.
            for word in chunk_owned.split_whitespace() {
                // This will count words exactly as they appear.
                *counts.entry(word.to_string()).or_insert(0) += 1;
            }
            counts
        });
        handles.push(handle);
    }

    // Join all threads and print their JSON dictionaries, one per line.
    for handle in handles {
        let counts = handle.join().expect("Thread panicked");
        // Create JSON string manually.
        // Note: the order of keys is arbitrary.
        let mut json = String::from("{");
        let mut first = true;
        for (key, count) in counts {
            if !first {
                json.push_str(", ");
            }
            // Escape double quotes in the key if any.
            let escaped_key = key.replace("\"", "\\\"");
            json.push_str(&format!("\"{}\": {}", escaped_key, count));
            first = false;
        }
        json.push('}');
        println!("{}", json);
    }
}

Check if all of the following conditions are true:

LLM Generation

Query

Below is some student's code. I asked them to parallelize some code. Do not judge if it is correct, I have test cases for that. Logic errors are okay. So are synatx errors. Just tell me if they tried to parallelize it. #!/usr/bin/env rust use std::collections::HashMap; use std::fs; use std::thread; fn main() { // Read the file into a String. Panic on error. let text = fs::read_to_string("/tmp/file.txt").expect("Failed to read /tmp/file.txt"); let total_len = text.len(); if total_len == 0 { // If file is empty, then print four empty JSON dictionaries. for _ in 0..4 { println!("{{}}"); } return; } // We want to divide the text into 4 roughly equal chunks. // To avoid cutting a word in half, we adjust the boundaries: let chunk_count = 4; let chunk_size = total_len / chunk_count; let mut offsets = Vec::with_capacity(chunk_count + 1); offsets.push(0); // For chunk boundaries at indices 1..(chunk_count), adjust forward to the next whitespace. for i in 1..chunk_count { let mut idx = i * chunk_size; // Ensure idx is at a valid UTF-8 boundary. while idx < total_len && !text.is_char_boundary(idx) { idx += 1; } // Now, if we are in the middle of a word (i.e. the character before idx is not whitespace), // then move forward until we pass the current word. if idx < total_len { // Check previous char if exists. if idx > 0 { let prev_char = text[..idx].chars().rev().next().unwrap(); if !prev_char.is_whitespace() { // We are in the middle of a word, so skip characters until we hit a whitespace. while idx < total_len { // Get the next char. let c = text[idx..].chars().next().unwrap(); idx += c.len_utf8(); if c.is_whitespace() { break; } } } } } offsets.push(idx); } offsets.push(total_len); // Now we have chunk boundaries in offsets: [start0, start1, start2, start3, end]. // Spawn 4 threads, each processing one chunk. let mut handles = Vec::new(); // We want to move the Arc'ed text to threads. Because our text is owned and immutable, // we can pass slices as &str since they have the same lifetime. for i in 0..chunk_count { let chunk = &text[offsets[i]..offsets[i+1]]; // Convert the chunk string slice into an owned String so the thread is independent. // (Alternatively, we might clone the &str reference if we used Arc but here cloning the chunk slice is simple.) let chunk_owned = chunk.to_string(); let handle = thread::spawn(move || { let mut counts: HashMap<String, usize> = HashMap::new(); // Split the chunk by whitespace. This preserves punctuation attached to words. for word in chunk_owned.split_whitespace() { // This will count words exactly as they appear. *counts.entry(word.to_string()).or_insert(0) += 1; } counts }); handles.push(handle); } // Join all threads and print their JSON dictionaries, one per line. for handle in handles { let counts = handle.join().expect("Thread panicked"); // Create JSON string manually. // Note: the order of keys is arbitrary. let mut json = String::from("{"); let mut first = true; for (key, count) in counts { if !first { json.push_str(", "); } // Escape double quotes in the key if any. let escaped_key = key.replace("\"", "\\\""); json.push_str(&format!("\"{}\": {}", escaped_key, count)); first = false; } json.push('}'); println!("{}", json); } } Answer either 'The student passes' or 'The student fails'.

Output

The student passes.

Substring Evaluation

Testing if the previous output contains the string student passes: True

Run Code Interpreter

Running the following program: 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
#!/usr/bin/env rust
use std::collections::HashMap;
use std::fs;
use std::thread;

fn main() {
    // Read the file into a String. Panic on error.
    let text = fs::read_to_string("/tmp/file.txt").expect("Failed to read /tmp/file.txt");

    let total_len = text.len();
    if total_len == 0 {
        // If file is empty, then print four empty JSON dictionaries.
        for _ in 0..4 {
            println!("{{}}");
        }
        return;
    }

    // We want to divide the text into 4 roughly equal chunks.
    // To avoid cutting a word in half, we adjust the boundaries:
    let chunk_count = 4;
    let chunk_size = total_len / chunk_count;
    let mut offsets = Vec::with_capacity(chunk_count + 1);
    offsets.push(0);

    // For chunk boundaries at indices 1..(chunk_count), adjust forward to the next whitespace.
    for i in 1..chunk_count {
        let mut idx = i * chunk_size;
        // Ensure idx is at a valid UTF-8 boundary.
        while idx < total_len && !text.is_char_boundary(idx) {
            idx += 1;
        }
        // Now, if we are in the middle of a word (i.e. the character before idx is not whitespace),
        // then move forward until we pass the current word.
        if idx < total_len {
            // Check previous char if exists.
            if idx > 0 {
                let prev_char = text[..idx].chars().rev().next().unwrap();
                if !prev_char.is_whitespace() {
                    // We are in the middle of a word, so skip characters until we hit a whitespace.
                    while idx < total_len {
                        // Get the next char.
                        let c = text[idx..].chars().next().unwrap();
                        idx += c.len_utf8();
                        if c.is_whitespace() {
                            break;
                        }
                    }
                }
            }
        }
        offsets.push(idx);
    }
    offsets.push(total_len);

    // Now we have chunk boundaries in offsets: [start0, start1, start2, start3, end].
    // Spawn 4 threads, each processing one chunk.
    let mut handles = Vec::new();
    // We want to move the Arc'ed text to threads. Because our text is owned and immutable,
    // we can pass slices as &str since they have the same lifetime.
    for i in 0..chunk_count {
        let chunk = &text[offsets[i]..offsets[i+1]];
        // Convert the chunk string slice into an owned String so the thread is independent.
        // (Alternatively, we might clone the &str reference if we used Arc but here cloning the chunk slice is simple.)
        let chunk_owned = chunk.to_string();
        let handle = thread::spawn(move || {
            let mut counts: HashMap<String, usize> = HashMap::new();
            // Split the chunk by whitespace. This preserves punctuation attached to words.
            for word in chunk_owned.split_whitespace() {
                // This will count words exactly as they appear.
                *counts.entry(word.to_string()).or_insert(0) += 1;
            }
            counts
        });
        handles.push(handle);
    }

    // Join all threads and print their JSON dictionaries, one per line.
    for handle in handles {
        let counts = handle.join().expect("Thread panicked");
        // Create JSON string manually.
        // Note: the order of keys is arbitrary.
        let mut json = String::from("{");
        let mut first = true;
        for (key, count) in counts {
            if !first {
                json.push_str(", ");
            }
            // Escape double quotes in the key if any.
            let escaped_key = key.replace("\"", "\\\"");
            json.push_str(&format!("\"{}\": {}", escaped_key, count));
            first = false;
        }
        json.push('}');
        println!("{}", json);
    }
}
And got the output: {"wisdom,": 1, "times,": 2, "foolishness,": 1, "best": 1, "it": 4, "age": 2, "worst": 1, "was": 4, "the": 4, "of": 4} {"belief,": 1, "the": 4, "season": 1, "Light,": 1, "it": 4, "epoch": 2, "incredulity,": 1, "of": 3, "was": 4} {"winter": 1, "everything": 1, "it": 2, "spring": 1, "hope,": 1, "was": 2, "of": 3, "season": 1, "the": 2, "despair,": 1, "we": 1, "had": 1, "before": 1, "Darkness,": 1} {"going": 2, "direct": 2, "we": 3, "nothing": 1, "were": 2, "all": 2, "had": 1, "before": 1, "Heaven,": 1, "us,": 2, "the": 1, "other": 1, "to": 1, "way": 1}

PyFunc

Resulting in output: [('Darkness,', 1), ('Heaven,', 1), ('Light,', 1), ('age', 2), ('all', 2), ('before', 2), ('belief,', 1), ('best', 1), ('despair,', 1), ('direct', 2), ('epoch', 2), ('everything', 1), ('foolishness,', 1), ('going', 2), ('had', 2), ('hope,', 1), ('incredulity,', 1), ('it', 10), ('nothing', 1), ('of', 10), ('other', 1), ('season', 2), ('spring', 1), ('the', 11), ('times,', 2), ('to', 1), ('us,', 2), ('was', 10), ('way', 1), ('we', 4), ('were', 2), ('winter', 1), ('wisdom,', 1), ('worst', 1)]

Equal Evaluation

Testing if the previous output equals the string [('Darkness,', 1), ('Heaven,', 1), ('Light,', 1), ('age', 2), ('all', 2), ('before', 2), ('belief,', 1), ('best', 1), ('despair,', 1), ('direct', 2), ('epoch', 2), ('everything', 1), ('foolishness,', 1), ('going', 2), ('had', 2), ('hope,', 1), ('incredulity,', 1), ('it', 10), ('nothing', 1), ('of', 10), ('other', 1), ('season', 2), ('spring', 1), ('the', 11), ('times,', 2), ('to', 1), ('us,', 2), ('was', 10), ('way', 1), ('we', 4), ('were', 2), ('winter', 1), ('wisdom,', 1), ('worst', 1)]: True

Final Answer: True