[LEGAL AI] Part 1-2: Planning AI Solutions for Accessibility

By Kael

Insights from My First Attempt: Lessons Learned

As I embark on developing the second version of my project, I find it essential to reflect on some critical takeaways from my initial experiences.

I recall a course that explored SLM fine-tuning and RAG for building a personalized AI assistant. While the course was informative, offering valuable insights into fine-tuning and optimization, it left a significant gap in its guidance—a gap I only fully realized during my own experimentation.

One of the course’s major oversights was the lack of detailed explanations on downloading and setting up the model, a foundational step in any AI project.

For this project, I chose LLAMA 3.1 as the backbone for training. There are multiple ways and platforms to access the model, but my experiments highlighted an important truth: the quality of your results heavily depends on how you approach this initial phase. No matter how much effort you put into the subsequent stages, neglecting the setup can significantly limit the final output.

Through trial and error, I discovered that Hugging Face offers a superior framework for downloading and fine-tuning compared to the main website. However, what struck me as a critical omission was the lack of guidance on obtaining the token required for the download—an absolutely essential step that was entirely absent from the course.

This omission left me questioning how other learners managed this part of the process. It became clear to me that selecting and managing the model in the early stages is a decisive factor, whether you’re working on SLMs or SSLMs. The choices you make here will directly impact the quality of your results, making it one of the most crucial steps in the entire workflow.

Looking back, I see how much this experience underscored the importance of meticulous planning and attention to detail, particularly during the model selection and setup phase—an area too vital to be overlooked.

Is a Token Required to Download Models from Hugging Face?

When downloading certain models (particularly large language models like LLAMA) via Hugging Face, users are required to create an account and agree to licensing terms, which may involve obtaining a token.

Fact: For models like LLAMA, distributed by Meta through Hugging Face, downloading does require submitting personal information, agreeing to the license, and obtaining an access token.
Source Verification: This process is documented on Hugging Face’s LLAMA model page, where the licensing and token requirements are outlined.

Personally,

I still find immense value in the old-school approach. I write things down and sketch ideas on paper. I dive headfirst into long, often overwhelming legal texts.

This blend of tradition and innovation inspired me. I started experimenting with personalized legal AI. I began with criminal law. Reading through the entire criminal code was like opening a treasure chest of surprises.

I kept asking myself, “Wait, this law actually exists?” It was a humbling, eye-opening experience. If criminal law is this fascinating, how much more must there be to uncover in other legal domains?

These realizations cemented a belief I’d been harboring: the future demands personalized legal AI. Not as a novelty, but as a necessity—a tool to make laws more accessible, understandable, and actionable for everyone.

The idea of creating a personalized legal AI service arose from recognizing that the legal field is vast. It is often overwhelming for individuals seeking information.

Even after immersing myself in just the criminal law statutes, I realized the complexity and the necessity for tools that can simplify legal jargon and provide personalized assistance. This project aims to bridge that gap by leveraging AI technologies.

Below is an image of the initial project proposal I created during the early stages of this work. Well, it’s not the latest version, I believe it serves as a good reference for understanding the foundation of the project and the thought process that went into it.

Attaching this older version gives a glimpse of how ideas evolved over time.

Outline of Workflow

Sorry, If you’re not Korean, It’s now easy to see right?

I made it and post a more concise version of the project plan, rewritten in English for clarity and broader accessibility. While it’s more streamlined than the original, it retains all the key elements and core ideas.

Starting with LLAMA 3.1: The Good, the Bad, and the “Why Did I Do That?”

When I began, I chose LLAMA 3.1 as my model. My initial enthusiasm was, let’s say, “unrefined.” I went full manual mode—downloading files, extracting them, and trying to make sense of the chaos. While it was a valuable lesson in patience, I eventually stumbled upon a much better method: downloading the model through Hugging Face, using the meta-llama/(model name) repository.

import transformers
import torch

model_id = “meta-llama/Llama-3.1-8B”

pipeline = transformers.pipeline(
“text-generation”, model=model_id, model_kwargs={“torch_dtype”: torch.bfloat16}, device_map=”auto”
)

pipeline(“Hey how are you doing today?”)

Now, things have evolved. Hugging Face requires you to “submit” your personal information to access LLAMA models. I’ve included for reference:

If that’s not your style, you can grab the models directly from Meta’s official site. But here’s my take: if you’re serious about fine-tuning and high-quality results, stick with Hugging Face.

Why? LLAMA’s base model isn’t great at processing Korean. Fine-tuning isn’t optional; it’s mandatory. When I used models directly downloaded from Meta, the results were, frankly, underwhelming. Hugging Face’s infrastructure, combined with its community resources, made a noticeable difference in performance and usability.

Project Overview

The project’s goal is to develop a legal AI service with a focus on Korean law, starting with criminal law. The service should understand and process Korean language effectively, providing accurate legal information and assistance.

Planning the Legal AI Service

A successful AI project requires meticulous planning. Below is the detailed breakdown of the planning process:

1. Defining Requirements

License Compliance: Ensure that the chosen AI models and datasets adhere to legal and ethical guidelines.
Model Selection: Initially, LLAMA 3.1 was chosen. However, it’s essential to evaluate if the model supports the target language (Korean in this case) and allows for necessary customizations.
Data Privacy: Compliance with data protection laws, especially when handling sensitive legal information.

2. Data Requirements

Language Proficiency: LLAMA models are not inherently proficient in Korean. Therefore, acquiring extensive Korean legal data is crucial.
Data Sources: Utilize reputable sources like AI Hub for abundant Korean language datasets, minimizing the need for manual data collection or web scraping.
Data Licensing: Ensure that all data used for training is properly licensed and permissible for use in AI training.

3. Hardware and Software Considerations

Hardware: High-performance GPUs are needed for training large models. In this case, an NVIDIA RTX 3090 was used.
Software: Decide on the AI frameworks and tools (e.g., huggingface lib, langchain, PyTorch, TensorFlow ….), along with libraries for natural language processing.

4. API Integration and Web Crawling

Registration and Permissions: If the service accesses external APIs or web data, ensure proper registrations and compliance with terms of service.
Legal Compliance: Special care must be taken when handling legal data to avoid unauthorized access or use.

5. Utilizing Open-Source Legal Data

Advantages: Open-source data can accelerate development and provide a robust foundation.
Verification: Always verify the accuracy and relevance of open-source legal data.

6. Data Preprocessing

Data preprocessing is critical for model performance, especially in specialized domains like law.

Steps Involved:

Text Normalization: Standardize text to a consistent format.
Morphological Analysis: Understand the structure of words to interpret meaning accurately.
Tokenization: Break down text into tokens (words, phrases) that the model can process. (Need to file proper Tokenizer that handles Korean well)
Stop Word Removal: Eliminate common words that don’t add significant meaning.
Named Entity Recognition (NER): Identify and categorize key entities (people, organizations, statutes). NER is especially important in legal contexts.
Sentence Segmentation and Structuring: Organize text into coherent sentences and paragraphs for better comprehension.

7. Fine-Tuning the Model

Specialized Training: Fine-tuning the AI model with the preprocessed legal data to improve understanding of legal terminology and concepts.
Multitask Learning: Allow the model to perform multiple tasks, enhancing its versatility.
Knowledge Distillation: If necessary, implement techniques to transfer knowledge from larger models to smaller ones for efficiency.

8. AI Model Training and Evaluation

Initial Training: Start with baseline training to observe how the model performs with legal data.
Hyperparameter Tuning: Adjust parameters like learning rate, batch size, and epochs for optimal performance.
Evaluation Metrics: Use metrics suitable for language models, such as accuracy, F1 score, and BLEU scores. For Korean language understanding, consider using the KLUE benchmark.

9. Performance Enhancement Techniques

Retrieval-Augmented Generation (RAG): Combine the language model with a retrieval system to improve accuracy by providing the model with relevant documents during inference.
Auto Chain-of-Thought (Auto-CoT): Enable the model to reason through problems step-by-step, improving its ability to handle complex queries.
Vector Databases: Use vector databases for efficient storage and retrieval of embeddings, enhancing the model’s ability to find relevant information.

10. Service Design and Deployment

Server Infrastructure: Set up scalable and reliable servers to handle user requests.
Database Design and Optimization: Design databases for storing user data, logs, and legal documents, ensuring quick access and security.

11. Security and Privacy Measures

Security Policies: Implement robust security protocols to protect user data and prevent unauthorized access.
Encryption: Use encryption for data at rest and in transit.
Compliance: Ensure adherence to data protection regulations like GDPR or local privacy laws.

12. UI/UX Design

User-Centric Design: Develop an intuitive interface that’s easy to navigate.
Accessibility: Ensure the service is accessible to users with disabilities.
Testing: Perform usability testing to identify and fix interface issues.

13. Service Deployment and Maintenance

Beta Testing: Launch a beta version to gather user feedback and identify bugs.
Monitoring: Continuously monitor the system for performance issues and errors.
Regular Updates: Keep legal data up-to-date to maintain the relevance and accuracy of the AI’s responses.

14. Continuous Improvement

User Feedback Integration: Use feedback to improve the AI’s capabilities and user experience.
Adaptation to Legal Changes: Regularly update the AI to reflect changes in laws and regulations.
Scaling: Plan for scaling the service to accommodate more users or additional features.

Visualizing the Plan: Mermaid Diagram

As previously mentioned, this is the English version : the planning process,

I converted the steps into a Mermaid diagram.

Important Considerations

Legal and Ethical Implications

Bias Mitigation: Training data often carries inherent biases, and addressing these is not just a technical challenge but an ethical one. For me, it’s about actively identifying these biases and implementing strategies—like balanced datasets or algorithmic fairness checks—to minimize their impact.
Transparency: Users deserve to know how their data is being used, plain and simple. Whether it’s through clear documentation or user-friendly explanations, ensuring transparency is a non-negotiable part of the process.
Accountability: No matter how sophisticated the system, human oversight must remain a priority. Building mechanisms that allow for intervention when necessary ensures the AI stays aligned with ethical standards and user trust.

These principles aren’t just theoretical—they’re embedded in every project I undertake, forming the foundation for AI that people can rely on.

Overcoming Challenges in Legal AI Development

Building a robust legal AI service isn’t without its challenges, but identifying them early is key to creating effective solutions.

Language Barriers
AI models often struggle with non-English languages like Korean, making proper fine-tuning and data preparation essential. This process involves curating high-quality datasets and employing techniques to bridge the gap in linguistic nuances, ensuring the AI understands and responds accurately in multiple languages.
Computational Resources
Training large AI models demands significant computational power, which can be a bottleneck for smaller teams or individuals. Optimizing code for efficiency and leveraging cloud-based solutions can help manage these demands without compromising performance.

Conclusion

Planning a legal AI service is a multifaceted endeavor that requires careful consideration of technical, legal, and ethical aspects. By methodically addressing each component—from defining initial requirements to planning for continuous improvement—you can lay a strong foundation for a successful AI project.

The legal field stands to benefit immensely from AI innovations, and with meticulous planning, we can create tools that not only enhance the efficiency of legal services but also make them more accessible to the public.

Final Thoughts

Planning a legal AI service is a multifaceted endeavor, requiring meticulous attention to technical, legal, and ethical considerations. By addressing these challenges systematically—from defining clear initial requirements to ensuring room for continuous improvement—you create a foundation for a successful and impactful AI project.

The legal field has much to gain from AI innovations. With careful planning and execution, we can develop tools that not only boost the efficiency of legal services but also make them more accessible and user-friendly for the public. The future of legal AI is full of possibilities, and with the right approach, we can turn these possibilities into tangible realities.

Embarking on the Journey of Personalized Legal AI

Diving into the development of a personalized legal AI service is a journey marked by both challenges and rewards. It demands not only a deep understanding of AI technologies but also a nuanced grasp of legal intricacies.

By blending these two complex domains with careful planning and meticulous execution, we can craft a service that transforms how legal assistance is delivered and utilized.

Such a service holds the potential to make legal systems more accessible, efficient, and tailored to the unique needs of individuals in ways we’ve only just begun to imagine.

Thank you for joining me on this journey through planning a legal AI service. If you have any questions or insights, feel free to leave a comment below. Let’s continue the conversation on how AI can transform the legal industry.

References and Resources

Hugging face

AI hub

META/LLAMA3.1

국가법령정보센터

personal info