Data science
Data science

Data science

A deep dive into how Amex's new Frontier Research Team is using AI and ML to build better modeling solutions - Tearsheet

American Express is harnessing AI and machine learning to enhance credit and risk management strategies.

fromiRunFar

1 hour ago

Data science

From Shakeout to Shakeup: Artificial Intelligence and Trail Running

AI is revolutionizing trail running by enhancing efficiency and task automation in the industry.

3 months ago

LLMs in Data Engineering: Not Just Hype, Here's What's Real | HackerNoon

Large Language Models are transforming data engineering by enhancing performance and operational efficiencies.

fromTearsheet

more#artificial-intelligence

Data science

A deep dive into how Amex's new Frontier Research Team is using AI and ML to build better modeling solutions - Tearsheet

American Express is harnessing AI and machine learning to enhance credit and risk management strategies.

fromiRunFar

1 hour ago

Data science

From Shakeout to Shakeup: Artificial Intelligence and Trail Running

AI is revolutionizing trail running by enhancing efficiency and task automation in the industry.

#machine-learning

fromeLearning Industry

Harnessing Educational Data Mining: A Guide For Instructional Designers

Educational Data Mining enhances instructional design by providing insights from large educational data, improving personalization and decision-making.

6 days ago

What Are AI Credits and How Can Data Scientists Use Them?

AI credits facilitate access to essential tools for data science teams, enhancing innovation and cost-effectiveness.

fromInfoQ

LLM and Generative AI for Sensitive Data - Navigating Security, Responsibility, and Pitfalls in Highly Regulated Industries

AI is significantly transforming various fields, including engineering, law, and healthcare, through innovative applications and responsible practices.

Big Data for the Data Science-Driven Manager 03- Apache Spark Explained for Managers

Apache Spark is crucial for efficiently processing large datasets in modern enterprises.

Data science

ODSC East 2025: Meet the Innovators at the AI Expo & Demo Hall

ODSC East 2025 will feature cutting-edge innovations and industry leaders in AI, data science, and machine learning.

Data science

The AI Engineer's Playbook: Mastering Vector Search & Management (Part 2) | HackerNoon

Vector embeddings require effective management systems to realize their potential in AI.

Vector Search & Management (VS&M) is crucial for value-driven data utilization.

fromeLearning Industry

Harnessing Educational Data Mining: A Guide For Instructional Designers

Educational Data Mining enhances instructional design by providing insights from large educational data, improving personalization and decision-making.

6 days ago

What Are AI Credits and How Can Data Scientists Use Them?

AI credits facilitate access to essential tools for data science teams, enhancing innovation and cost-effectiveness.

fromInfoQ

LLM and Generative AI for Sensitive Data - Navigating Security, Responsibility, and Pitfalls in Highly Regulated Industries

AI is significantly transforming various fields, including engineering, law, and healthcare, through innovative applications and responsible practices.

Big Data for the Data Science-Driven Manager 03- Apache Spark Explained for Managers

Apache Spark is crucial for efficiently processing large datasets in modern enterprises.

Data science

ODSC East 2025: Meet the Innovators at the AI Expo & Demo Hall

ODSC East 2025 will feature cutting-edge innovations and industry leaders in AI, data science, and machine learning.

Data science

The AI Engineer's Playbook: Mastering Vector Search & Management (Part 2) | HackerNoon

Vector embeddings require effective management systems to realize their potential in AI.

Vector Search & Management (VS&M) is crucial for value-driven data utilization.

more#machine-learning

#data-engineering

Data science

Spark Scala Exercise 23: Working with Delta Lake in Spark ScalaACID, Time Travel, and Upserts

Delta Lake enhances data reliability and governance for data lakes by integrating warehouse features.

Data science

Spark Scala Exercise 10: Handling Nulls and Data CleaningFrom Raw Data to Analytics-Ready

Effective data cleaning is essential in data engineering to prevent downstream issues caused by nulls.

fromChannelPro

Data science

Datatonic expands global services with Syntio acquisition

Datatonic expands its services by acquiring data engineering firm Syntio, enhancing global reach and expertise in AI solutions.

fromTechzine Global

Data science

Datatonic acquires Syntio and strengthens expertise in data engineering

Datatonic's acquisition of Syntio enhances its data consultancy with increased capabilities in data engineering and expanded service offerings.

Data science

Tired of Copy-Pasting Hive Output? This PySpark Hack Fixes It | HackerNoon

Automating CSV export from Hive or Impala output is essential for efficient data engineering tasks.

9 months ago

Data science

What is Microsoft Fabric? A big tech stack for big data

Microsoft Fabric is a comprehensive cloud-based platform for data analytics that integrates various Microsoft tools.

Data science

Spark Scala Exercise 23: Working with Delta Lake in Spark ScalaACID, Time Travel, and Upserts

Delta Lake enhances data reliability and governance for data lakes by integrating warehouse features.

Data science

Spark Scala Exercise 10: Handling Nulls and Data CleaningFrom Raw Data to Analytics-Ready

Effective data cleaning is essential in data engineering to prevent downstream issues caused by nulls.

fromChannelPro

Data science

Datatonic expands global services with Syntio acquisition

Datatonic expands its services by acquiring data engineering firm Syntio, enhancing global reach and expertise in AI solutions.

fromTechzine Global

Data science

Datatonic acquires Syntio and strengthens expertise in data engineering

Datatonic's acquisition of Syntio enhances its data consultancy with increased capabilities in data engineering and expanded service offerings.

Data science

Tired of Copy-Pasting Hive Output? This PySpark Hack Fixes It | HackerNoon

Automating CSV export from Hive or Impala output is essential for efficient data engineering tasks.

9 months ago

Data science

What is Microsoft Fabric? A big tech stack for big data

Microsoft Fabric is a comprehensive cloud-based platform for data analytics that integrates various Microsoft tools.

more#data-engineering

#performance-optimization

1 day ago

Data science

Transforming Business Through SAP Innovation by Nagender Yadav | HackerNoon

Nagender Yadav transforms SAP implementations, significantly reducing costs and timelines with pioneering methods.

fromTalkpython

1 day ago

Data science

The PyArrow Revolution

PyArrow optimizes performance for data analysis in Python, positioning itself as a critical backend for Pandas.

The integration of PyArrow into Pandas marks a significant shift in data science practices.

Data science

Spark Scala Exercise 22: Custom Partitioning in Spark RDDsLoad Balancing and Shuffle

Implementing a custom partitioner in Spark helps manage load balance and optimize data distribution.

Data science

Spark Scala Exercise 22: Custom Partitioning in Spark RDDsLoad Balancing and Shuffle

Implementing a custom partitioner in Spark Scala enhances control over data distribution, improves performance in various scenarios, and optimizes task execution.

fromTalkpython

1 day ago

The PyArrow Revolution

PyArrow optimizes performance for data analysis in Python, positioning itself as a critical backend for Pandas.

The integration of PyArrow into Pandas marks a significant shift in data science practices.

Data science

Spark Scala Exercise 22: Custom Partitioning in Spark RDDsLoad Balancing and Shuffle

Implementing a custom partitioner in Spark helps manage load balance and optimize data distribution.

more#performance-optimization

Data science

Spark Scala Exercise 22: Custom Partitioning in Spark RDDsLoad Balancing and Shuffle

Implementing a custom partitioner in Spark Scala enhances control over data distribution, improves performance in various scenarios, and optimizes task execution.

#biomedical-text-mining

Data science

Future Perspectives in the Era of Large Language Models, and References | HackerNoon

Large language models necessitate robust evaluation benchmarks for biomedical text mining.

Future challenges should focus on multimodal data integration in biomedical research.

Data science

The Impact of Community Challenges on Biomedical Text Mining Research | HackerNoon

Community challenges have greatly advanced biomedical text mining by offering benchmarks and fostering collaboration.

Data science

Future Perspectives in the Era of Large Language Models, and References | HackerNoon

Large language models necessitate robust evaluation benchmarks for biomedical text mining.

Future challenges should focus on multimodal data integration in biomedical research.

more#biomedical-text-mining

Data science

The Impact of Community Challenges on Biomedical Text Mining Research | HackerNoon

Community challenges have greatly advanced biomedical text mining by offering benchmarks and fostering collaboration.

4 days ago

Data science

Databricks to infuse $250M to double its R&D staff in India this year

The company is investing $250 million to enhance its Data + AI academy in India.

The Ultimate Data Visualization Handbook for Designers

Data visualization is crucial for making sense of the vast amounts of data generated daily.

Clarity and simplicity are essential in effective data visualization design.

Choosing the right methods and tools is fundamental in the visualization process.

Data science

How GenAIs build diverging color schemes

Generative AI can effectively create tailored diverging data color schemes for data visualization based on specific hues like Mocha Mousse.

Data science

How GenAIs build diverging color schemes

Generative AI can create customized diverging data color schemes for visualization using specific Pantone colors.

fromErik Marsja

Data science

How to Extract GPS Coordinates from a Photo: The USAID Mystery

Photographs today capture hidden data like geolocation, revealing where they were taken.

4 days ago

Data science

How To Make a Pie Chart in Google Sheets (Step-by-Step)

Pie charts provide a simple way to visualize data for easy analysis.

Google Sheets enables users to create and customize pie charts effortlessly.

Data science

How to Make a Line Graph or Chart in Google Sheets

Creating line charts in Google Sheets can be overwhelming for beginners, but step-by-step guidance can assist in the process.

fromUX Magazine

The Ultimate Data Visualization Handbook for Designers

Data visualization is crucial for making sense of the vast amounts of data generated daily.

Clarity and simplicity are essential in effective data visualization design.

Choosing the right methods and tools is fundamental in the visualization process.

Data science

How GenAIs build diverging color schemes

Generative AI can effectively create tailored diverging data color schemes for data visualization based on specific hues like Mocha Mousse.

Data science

How GenAIs build diverging color schemes

Generative AI can create customized diverging data color schemes for visualization using specific Pantone colors.

fromErik Marsja

Data science

How to Extract GPS Coordinates from a Photo: The USAID Mystery

Photographs today capture hidden data like geolocation, revealing where they were taken.

4 days ago

Data science

How To Make a Pie Chart in Google Sheets (Step-by-Step)

Pie charts provide a simple way to visualize data for easy analysis.

Google Sheets enables users to create and customize pie charts effortlessly.

Data science

How to Make a Line Graph or Chart in Google Sheets

Creating line charts in Google Sheets can be overwhelming for beginners, but step-by-step guidance can assist in the process.

more#data-visualization

Ignoring citizens' privacy to build a centralized database and track people

The consolidation of government data for tracking immigrants raises concerns about privacy and accuracy.

The Trump administration's executive order endorses data sharing across federal agencies.

fromwww.theguardian.com

Data science

Predictive policing has prejudice built in | Letters

Data automation is perpetuating discrimination against marginalized communities and lacks evidence for preventing crime.

fromThe Verge

Data science

DOGE staffer 'Big Balls' has access to immigration agency's data

USCIS granted DOGE staffers access to sensitive immigration data without clear justification.

fromFlowingData

Ignoring citizens' privacy to build a centralized database and track people

The consolidation of government data for tracking immigrants raises concerns about privacy and accuracy.

The Trump administration's executive order endorses data sharing across federal agencies.

fromwww.theguardian.com

Data science

Predictive policing has prejudice built in | Letters

Data automation is perpetuating discrimination against marginalized communities and lacks evidence for preventing crime.

fromThe Verge

Data science

DOGE staffer 'Big Balls' has access to immigration agency's data

USCIS granted DOGE staffers access to sensitive immigration data without clear justification.

Datadog acquires Metaplane, a data problem-solver

Datadog acquires Metaplane to enhance data observability capabilities with AI-driven insights.

Data science

Database Revolution Series: A Modern Guide to Data Management

Modern data management is essential for real-time analytics and AI support.

Traditional relational databases struggle with the variety of today's data types.

fromTechzine Global

6 days ago

Datadog acquires Metaplane, a data problem-solver

Datadog acquires Metaplane to enhance data observability capabilities with AI-driven insights.

Data science

Database Revolution Series: A Modern Guide to Data Management

Modern data management is essential for real-time analytics and AI support.

Traditional relational databases struggle with the variety of today's data types.

more#ai

3 months ago

Data science

How AI Models Gender and Sexual Orientation | HackerNoon

Language models reveal socio-psychological harms by misrepresenting identities, emphasizing the need for more inclusive AI approaches.

fromState of the Planet

A New Interactive Tool Models Natural Hazards Fueled by Climate Change

Climate change is worsening the severity and frequency of extreme weather events, necessitating better predictive models for disaster preparedness.

fromFlowingData

List of NOAA datasets to be discontinued

Urgent download of NOAA datasets is recommended as many are set for retirement in the coming weeks.

fromFlowingData

Halt of data collection that measures American society

The removal of federal datasets hinders the government's ability to accurately assess societal issues.

fromwww.nature.com

Data science

Author Correction: Structure of the human dopamine transporter in complex with cocaine

Corrections made to the structure analysis of hDAT with no impact on overall conclusions.

#data-strategy

Data science

Database Revolution Series: A Modern Guide to Data Management

Multi-model and cloud-native databases are transforming data management for businesses.

Data science

Data mesh vs. data fabric vs. data virtualization: There's a difference

Data mesh is a decentralized approach where domain experts manage their own data, treating it as a product.

Data science

Database Revolution Series: A Modern Guide to Data Management

SQL databases manage structured data efficiently, while NoSQL is ideal for unstructured data.

Data science

Database Revolution Series: A Modern Guide to Data Management

Multi-model and cloud-native databases are transforming data management for businesses.

Data science

Data mesh vs. data fabric vs. data virtualization: There's a difference

Data mesh is a decentralized approach where domain experts manage their own data, treating it as a product.

Data science

Database Revolution Series: A Modern Guide to Data Management

SQL databases manage structured data efficiently, while NoSQL is ideal for unstructured data.

Data science

Redis 8 Targets AI Applications with New Data Type for Vector Similarity

Redis has introduced Vector Sets, a new data type enhancing AI applications through vector similarity.

Vector Sets enable semantic searches and support filtered capabilities for AI systems.

Data science

Scientists Built a Knowledge Graph for Materials-And You Can Actually Use It | HackerNoon

The article discusses a method for representing material relationships using triples in a graph database.

The use of FMKG and Neo4j significantly improves data management and retrieval for material sciences.

fromInfoQ

Data science

Redis 8 Targets AI Applications with New Data Type for Vector Similarity

Redis has introduced Vector Sets, a new data type enhancing AI applications through vector similarity.

Vector Sets enable semantic searches and support filtered capabilities for AI systems.

Data science

Scientists Built a Knowledge Graph for Materials-And You Can Actually Use It | HackerNoon

The article discusses a method for representing material relationships using triples in a graph database.

The use of FMKG and Neo4j significantly improves data management and retrieval for material sciences.

Data science

Database Revolution Series: A Modern Guide to Data Management

Modern data management solutions are essential due to the exponential growth of data and the inadequacies of traditional databases.

Data science

Database Revolution Series: A Modern Guide to Data Management

Modern data management solutions are essential due to the limitations of traditional databases in handling diverse and unstructured data.

Data science

Database Revolution Series: A Modern Guide to Data Management

The cloud revolution impacts how applications are designed and deployed, crucially through serverless computing and NewSQL databases.

Data science

Database Revolution Series: A Modern Guide to Data Management

Time-Series Databases and Vector Databases are essential for managing specialized data types effectively.

Data science

SQL vs. NoSQL Explained: When to Use Which and Why It Matters to Modern Data Management

SQL databases manage structured data using tables and ensure data integrity through ACID properties.

Data science

SQL vs. NoSQL Explained: When to Use Which and Why It Matters to Modern Data Management

SQL and NoSQL databases are essential for modern data management.

SQL databases utilize predefined schemas and provide robust data integrity through ACID properties.

Database Revolution Series: A Modern Guide to Data Management

Modern data management solutions are essential due to the exponential growth of data and the inadequacies of traditional databases.

Database Revolution Series: A Modern Guide to Data Management

Modern data management solutions are essential due to the limitations of traditional databases in handling diverse and unstructured data.

Database Revolution Series: A Modern Guide to Data Management

The cloud revolution impacts how applications are designed and deployed, crucially through serverless computing and NewSQL databases.

Database Revolution Series: A Modern Guide to Data Management

Time-Series Databases and Vector Databases are essential for managing specialized data types effectively.

SQL vs. NoSQL Explained: When to Use Which and Why It Matters to Modern Data Management

SQL databases manage structured data using tables and ensure data integrity through ACID properties.

SQL vs. NoSQL Explained: When to Use Which and Why It Matters to Modern Data Management

SQL and NoSQL databases are essential for modern data management.

SQL databases utilize predefined schemas and provide robust data integrity through ACID properties.

Data science

Using the Excel DAYS360 Function for Financial Analysis- A Guide | HackerNoon

The DAYS360 function calculates the number of days between dates based on a 360-day year, essential for financial calculations.

2 years ago

Data science

How to Use Excel DATE Function? -> Excel 24x7 | HackerNoon

The Excel DATE function creates valid dates using year, month, and day numbers for reliable date management.

1 month ago

Data science

Using the Excel DAYS360 Function for Financial Analysis- A Guide | HackerNoon

The DAYS360 function calculates the number of days between dates based on a 360-day year, essential for financial calculations.

2 years ago

Data science

How to Use Excel DATE Function? -> Excel 24x7 | HackerNoon

The Excel DATE function creates valid dates using year, month, and day numbers for reliable date management.

Data science

Handling Large Data Volumes (100GB-1TB) in Scala with Apache Spark

Apache Spark is essential for processing large datasets due to memory constraints and scalability of traditional tools.

Data science

Word Count Program

The Word Count program effectively demonstrates word counting using distributed computing frameworks.

fromLogRocket Blog

Data science

Use TypeScript instead of Python for ETL pipelines - LogRocket Blog

Building an ETL pipeline in TypeScript enhances type safety and maintainability while processing data from various sources.

Data science

Handling Large Data Volumes (100GB-1TB) in Scala with Apache Spark

Apache Spark is essential for processing large datasets due to memory constraints and scalability of traditional tools.

Data science

Word Count Program

The Word Count program effectively demonstrates word counting using distributed computing frameworks.

fromLogRocket Blog

Data science

Use TypeScript instead of Python for ETL pipelines - LogRocket Blog

Building an ETL pipeline in TypeScript enhances type safety and maintainability while processing data from various sources.

Data science

Salesforce bets AI agents can solve business leaders' struggles with data | MarTech

Business leaders face increasing pressure to utilize data effectively, despite declining confidence in data relevance and accuracy.

Data science

Spark Scala Exercise 8: Working with Date-Time in SparkExtract, Transform, and Analyze

Date and time operations are vital for analysis in various sectors, enabling insights into trends and customer behavior.

Data science

Spark Scala Exercise 7: Advanced Group By and Aggregations (with Rollup, Cube, and Multi-level

Advanced grouping techniques in Spark Scala enhance OLAP-style reporting for detailed analysis across industries.

fromMarTech

Salesforce bets AI agents can solve business leaders' struggles with data | MarTech

Business leaders face increasing pressure to utilize data effectively, despite declining confidence in data relevance and accuracy.

Data science

Spark Scala Exercise 8: Working with Date-Time in SparkExtract, Transform, and Analyze

Date and time operations are vital for analysis in various sectors, enabling insights into trends and customer behavior.

Data science

Spark Scala Exercise 7: Advanced Group By and Aggregations (with Rollup, Cube, and Multi-level

Advanced grouping techniques in Spark Scala enhance OLAP-style reporting for detailed analysis across industries.

Business leaders are having a crisis of confidence over data literacy

Business leaders feel pressured to utilize data for decisions, but face obstacles like lack of data trust and literacy.

fromEntrepreneur

Data science

Lead with Insight Using These 5 Success Strategies | Entrepreneur

Organizations struggle with becoming data-driven primarily due to cultural challenges rather than technological ones.

Establishing clear strategic goals is essential before adopting advanced analytics and AI.

Database Revolution Series: A Modern Guide to Data Management

Serverless computing and NewSQL databases revolutionize application development, enhancing scalability and simplifying the development process.

#spark

Data science

Spark Scala Exercise 4: DataFrame Schema Exploration (with Case Classes)

Understand how Spark infers schemas and the importance of Scala case classes for type safety.

Data science

Spark Scala Exercise 22: Custom Partitioning in Spark RDDsLoad Balancing and Shuffle

Custom partitioners in Spark Scala enable optimal control over data distribution for RDDs.

Data science

Spark Scala Exercise 4: DataFrame Schema Exploration (with Case Classes)

Understand how Spark infers schemas and the importance of Scala case classes for type safety.

Data science

Spark Scala Exercise 22: Custom Partitioning in Spark RDDsLoad Balancing and Shuffle

Custom partitioners in Spark Scala enable optimal control over data distribution for RDDs.

Hedge funds are scrambling to get tariff data

Hedge funds are seeking country-level data to assess the impact of President Trump's tariff policies on the global economy.

1 month ago

Data science

Transforming Health Insurance with AI-Driven Business Analytics: A Case Study in Digital Excellence | HackerNoon

AI-powered analytics is revolutionizing health insurance by enhancing risk assessment and claims management.

Ruchi Mangharamani leads initiatives that improve decision-making through predictive analytics and cost optimization.

fromBusiness Insider

Hedge funds are scrambling to get tariff data

Hedge funds are seeking country-level data to assess the impact of President Trump's tariff policies on the global economy.

1 month ago

Data science

Transforming Health Insurance with AI-Driven Business Analytics: A Case Study in Digital Excellence | HackerNoon

AI-powered analytics is revolutionizing health insurance by enhancing risk assessment and claims management.

Ruchi Mangharamani leads initiatives that improve decision-making through predictive analytics and cost optimization.

How to Process Large Files in Data Indexing Systems | HackerNoon

Efficiently processing large files in data indexing pipelines requires managing processing granularity and balancing commit frequency to optimize performance and recoverability.

fromNextgov.com

Top oversight Dem files resolution to demand answers from DOGE on AI use

Rep. Melanie Stansbury introduced a resolution demanding the Trump administration disclose details on Elon Musk's unit's use of federal data and AI.

fromAri Lamstein